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AVANT-PROPOS 


Richard  Bellman,  un  des  math^maticiens  les  plus  f§conds  et  les  plus 
renommes  des  Etats  Unis,  a  apporte  des  contributions  majeures  aux  mathema- 
tiques  pures  et  5  de  nombreux  domaines  d'applications  :  sciences  de  l'ing§- 
nieur,  economie,  m§decine,  gnergie,  gestion  des  ressources  en  eau,  physique 
mathematique,  recherche  operationnelle,  sciences  de  la  gestion,  psychologie 
et  sociologie.  Une  telle  variety  des  domaines  abord§s  et  des  moyens  mis  en 
oeuvre  pour  approfondir  ces  domaines  avec  une  telle  penetration  se  rencontre 
rarement  en  science. 

Tout  au  long  du  developpement  de  son  oeuvre,  i’  eut  un  grand  nombre 
d'amis,  d'eieves  et  de  correspondents  portSs  vers  les  memes  centres  d’interet. 
Parmi  eux,  apres  la  disparition  du  Professeur  Bellman,  un  groupe  de  scienti- 
fiques  des  Etats  Unis  s'est  efforcS  de  perpStuer  son  Ecole.  Dans  ce  but  ils 
ont  propose  d' organiser  un  Colloque  annuel  ou  bi -annuel  :  le  Bellman  Continuum. 
Cc  Colloque  devait  etre  de  nature  interdisciplinaire,  comme  I'etait  1 'oeuvre 
de  Richard  Bellman. 

Le  premier  congres  s'est  tenu  5  I'Universite  du  Michigan,  Ann  Arbor, 
Michigan,  en  1985  et  le  second  a  ete  accueilli  par  I'Institut  de  Technologie 
de  Georgie,  Atlanta,  Georgia,  en  1986.  Les  organisateurs  ont  pense  que  la 
France  serait  un  des  pays  les  mieux  adaptSs  &  la  tenue  du  troisieme  congres 
pour  des  raisons  de  caractgre  3  la  fois  scientifique  et  gSographique  : 

Richard  Bellman  6tait  tr§s  populaire  en  Europe.  De  plus,  un  argument  important 
pour  ce  choix  6tait  le  fait  que  la  huiti&ne  Conf§rence  Internationale  Analyse 
et  Optimisation  des  SystSmes  de  I'INRIA  devait  se  tenir  a  Antibes  du  8  au  10 
Juin  1988.  Ceia  fournissait  I'occasion  idfiale  de  profiter  de  la  presence  en 
un  mfime  lieu  d'un  grand  nombre  de  spScialistes  venant  de  tous  les  points  du 
monde  pour  organiser  une  petite  conference  permettant  un  echange  d'id6es  assez 
informel . 


Dans  1es  deux  premiers  congrds,  1e  programme  avait  §t6  dictg  par  1a 
nature  interdisciplinaire  du  Collogue  avec  des  sujets  dgfinis  suivant  les 
interSts  des  participants.  Les  th&nes  unificateurs  fetaient  1‘id§ologie  scien- 
tifique  et  les  techniques  mathdnatiques  p1ut6t  que  les  domaines  d'etude 
spScifiques.  Dans  ce  troisiSme  congrSs,  pour  des  raisons  scientifiques  §vi- 
dentes,  I'ensemble  des  sujets  abordfis  a  §t§  d§1ib§r§ment  restraint,  ayant  en 
vue  1e  fait  que  ces  sujets  pourraient  changer  d'un  congrSs  au  suivant.  Les 
sujets  mentionn^s  ci-dessous,  choisis  dans  des  domaines  oQ  la  recherche  est 
trds  active  et  pleine  de  promesses,  ont  §t§  s§lectionn§s  : 

Modelisation  et  commande  en  Economie  et  en  Sciences  Sociales. 

Commande  des  syst^mes  dynamiques  incertains. 

Commande  et  filtrage  nonlin§aire  des  processus  quantiques. 

Modelisation  et  commande  des  systdmes  biologiques. 

Les  Confferenciers  d'Ouverture  de  Sessions  sont  les  Professeurs 

R. E.  KALMAN,  University  of  Florida,  U.S.A.,  et  Technische  Hochschule, 

Zurich,  Suisse 

G.  LEITMANN,  University  of  California,  Berkeley,  U.S.A. 

S.  MITTER,  Massachusetts  Institute  of  Technology,  U.S.A. 

Ini ti element,  notre  intention  6tait  de  r§unir  un  petit  nombre  de  sp§cia- 
listes  sur  la  base  d' invitations .  Cependant,  les  r§ponses  3  notre  annonce 
prSliminaire  surpassSrent  notre  estimation  la  plus  optimiste  de  1 'enthousiasme 
des  chercheurs  dans  ces  domaines.  Par  la  suite,  nous  d§cid5mes  d'6diter  les 
Actes  de  ce  Collogue  sous  la  forme  d'un  livre  rfeunissant  les  conferences  sur 
invitation  et  certains  des  rapports  destines  3  la  presentation  de  travaux 
recents,  soumis  au  Comite  d' Organisation.  Ce  livre  sera  publie  aprSs  le 
congres  par  SPRINGER-VERLAG  dans  la  Serie  "Lecture  Notes  in  Control  and 
Information  Sciences".  Les  manuscrits  contenus  dans  le  present  fascicule  sont 
les  resumes  ou  les  textes  integraux  de  tous  les  papiers  en  notre  possession 
au  moment  du  congres.  Pour  cheque  theme,  dans  toute  la  mesure  du  possible, 
ils  sont  presentes  dans  I'ordre  oO  ils  se  trouvent  dans  le  Programne. 


Ce  Colloque  n' aural t  pu  avoir  lieu  sans  1e  soutien  technique  et 
financier  de  TINRIA  3  qui  nous  exprimons  notre  gratitude.  En  particulier, 
qu'il  me  soit  permis  de  remercier  ici  son  President  le  Professeur  Alain 
BENSOUSSAN,  le  Directeur  du  Centre  de  Recherche  de  1 ' INRIA-SOPHIA  ANTIPOLIS 
le  Professeur  Pierre  BERNHARD,  et  le  Directeur  des  Relations  Internationales 
et  Industrielles  Georges  NISSEN.  J'adresse  personnel lement  des  remercienents 
tout  particuliers  5  Th^rfise  BRICHETEAU  qui,  3  la  tfite  du  Service  des  Relations 
Exterieures  de  I'lNRIA,  a  pris  soin  si  efficacement  de  la  multitude  des 
probl&nes  d'organisation,  et  nous  a  fait  profiter  de  sa  grande  experience. 

Nous  somties  tr3s  reconnaissants  3  Catherine  JUNCKER  qui  a  pris  en  main  de 
fagon  experte  I'organisation  du  congrSs  3  Sophia  et  son  implantation  sur  le 
site.  Je  suis  redevable  envers  les  Secretaires  Scientifiques  de  la  huiti§me 
Conference  Analyse  et  Optimisation  des  SystSmes  de  I’lNRIA  de  leur  aide, 
notamment  envers  Frederic  BONNANS  pour  ses  conseils  eclair§s  dans  la  prepara¬ 
tion  du  Colloque.  Des  remerciements  tout  particuliers  sont  dQs  3  Gilbert 
MALLET  qui  a  edite  ces  "Preprints".  Finalement,  et  non  les  moindres,  nos 
remerciements  s'adressent  3  toutes  les  secretaires  et  les  techniciens  qui 
ont  contribuS  au  succ6s  du  troisiSme  Bellman  Continuum,  aux  diffSrentes 
etapes  et  aux  diffSrents  niveaux  de  son  organisation. 

Ce  Colloque  international  a  aussi  regu  le  soutien  financier  de 
1 'University  Tiris  7,  I'un  des  organisateurs  du  congres,  de  TAssociation 
Frangaise  pour  la  CybernStique  Economique  et  Technique  (AFCET),  du  Centre 
National  de  la  Recherche  Scientifique  (CNRS),  de  I'European  Research  Dffice 
United  States  Army,  du  MinistSre  des  Affaires  Etrang3res,  du  Minist§re  de 
I'Education  Nationale,  du  MinistSre  de  la  Recherche  et  de  I'Enseignement 
Supferieur,  de  1 'United  Nations  Educational  Scientific  and  Cultural  Organization 
(UNESCO).  D'autres  subventions  ont  §ty  attributes  aux  participants  par  divers 
organismes  mentionnts  stpartment,  auxquels,  comme  aux  organismes  dont  nous 
venons  de  dresser  la  liste,  nous  exprimons  notre  gratitude. 


Austin  BLAQUIERE  (President) 


FOR  EHORO 


Richard  Bellman,  a  most  prolific  and  renowned  mathematician  of  the  United 
States,  has  made  major  contributions  In  pure  mathematics  and  in  numerous  areas 
of  applications  :  engineering,  economics,  medicine,  energy,  water  resources, 
mathematical  physics,  operations  research,  management  sciences,  psychology  and 
sociology.  This  breadth  of  Interests  and  this  ability  to  contribute  to  so 
many  fields  at  such  a  high  level  Is  rare  Indeed. 


Throughout  his  years  In  science,  he  had  a  large  number  of  scientific 
friends,  students  and  followers.  Among  them,  after  Professor  Bellman  has 
passed  away,  a  group  of  scientists  of  the  United  States  has  attempted  to 
preserve  his  School.  As  a  mechanism  for  achieving  this  goal,  they  suggested 
an  annual  or  biennial  workshop  ;  the  Bellmn  Continuum.  This  workshop  was 
envisioned  as  being  Interdisciplinary  In  nature,  as  the  achievement  of 
Richard  Bellman  was. 


The  first  meeting  was  held  at  the  University  of  Michigan,  Ann  Arbor, 
Michigan,  In  1985  and  the  second  was  hosted  by  the  Georgia  Institute  of 
Technology,  Atlanta,  Georgia,  in  1986.  The  organizers  thought  that  France 
could  be  a  nice  place  for  the  third  meeting  from  both  scientific  and  geogra¬ 
phical  points  of  view  :  Richard  Bellman  was  very  popular  In  Europe.  Also,  a 
strong  motivation  for  this  choice  was  the  fact  that  the  eighth  International 
Conference  Analysis  and  Optimization  of  Systems  of  INRIA  was  to  be  held  In 
Antibes  on  June  8-10,  1988.  It  provided  an  Ideal  opportunity  for  taking 
advantage  of  the  presence  of  a  large  number  of  specialists  from  all  parts 
of  the  world  to  organize  a  small  conference  where  a  free  exchange  of  Ideas 
could  take  place. 


/ 


In  the  two  first  meetings,  the  program  has  been  dictated  by  the  Inter¬ 
disciplinary  nature  of  the  workshop  with  topics  defined  by  the  Interest  of 
the  participants.  The  unifying  theme  Included  scientific  Ideology  and 
mathematical  tools  rather  than  specific  fields  of  study.  In  this  third  one, 
for  evident  scientific  purposes  the  subject  matter  to  be  treated  has  been 
limited,  having  In  view  the  fact  that  the  areas  defined  below  could  change 
from  one  meeting  to  the  next.  The  following  topics,  chosen  In  areas  where 
research  Is  very  active  and  promising.  In  directions  opened  and  explored  by 
Richard  Bellman,  have  been  selected  : 

Models  and  Control  Policies  In  Economics  and  Social  Systems. 

Control  of  Uncertain  Dynamical  Systems. 

Control  and  Nonlinear  Filtering  of  Quantum  Mechanical  Processes. 

Models  and  Control  Policies  for  Biological  Systems. 

The  Key-note  Speakers  are 

Professor  R.E.  KALMAN,  University  of  Florida,  U.S.A.,  and  Technische 
Hochschule,  Zurich,  Switzerland 

Professor  G.  LEITMANN,  University  of  California,  Berkeley,  U.S.A. 

Professor  S.  MITTER,  Massachusetts  Institute  of  Technology,  U.S.A. 

Originally,  It  was  thought  that  a  gathering  of  a  small  number  of 
specialists  on  an  Invited  basis  was  sufficient  for  the  purpose.  However, 
the  responses  to  our  Initial  announcement  surpassed  our  most  optimistic 
estimate  of  the  enthusiasm  of  workers  In  these  areas.  Subsequently,  It  was 
decided  that  we  edit  the  Proceedings  of  this  workshop  as  a  book  containing 
all  the  Invited  papers  and  selected  contributed  papers  submitted  to  the 
workshop.  This  book  will  be  published  after  tlw  meeting  by  SPRINGER-VERLAG 
In  the  Series  "Lecture  Notes  In  Control  and  Information  Sciences".  The 
manuscripts  contained  In  the  present  Preprints  are  extended  summaries  or  full 
text  of  all  papers  available  from  authors  at  the  time  of  the  meeting. 


This  workshop  could  not  have  taken  place  without  the  technical  and 
financial  assistance  of  INRIA  to  whom  we  express  our  gratitude.  In  particular 
I  would  like  to  take  this  opportunity  to  thank  his  President  Professor  Alain 
BENSOUSSAN.  the  Director  of  the  INRIA-SOPHIA  ANTIPOLIS  Research  Center 
Professor  Pierre  BERNHARD,  and  the  Director  of  Industrial  and  International 
Relations  Georges  NISSEN.  I  personnally  address  special  thanks  to  Therfise 
BRICHETEAU  who,  at  the  head  of  the  Service  des  Relations  Exterleures  of  INRIA, 
took  care  of  all  the  myriad  details  of  organization  so  efficiently  and  ably. 

Me  are  most  grateful  to  the  expert  assistance  of  Catherine  JUNCKER  who  took 
care  of  the  organization  at  Sophia  and  of  the  local  arrangements.  I  am 
Indebted  to  the  Scientific  Secretaries  of  the  eighth  INRIA  Conference 
Analysis  and  Optimization  of  Systems  for  their  help.  In  particular  to 
Frederic  BONNANS  for  valuable  advice  In  the  preparation  of  the  workshop. 

Especial  thanks  go  to  Gilbert  MALLET  who  edited  these  Preprints.  Last  but 
not  least,  our  thanks  go  to  all  the  secretaries  and  technicists  who  handled 
the  many  problems  at  each  step  and  at  each  level  and  contributed  to  the 
success  of  the  third  Bellman  Continuum. 

This  International  workshop  was  also  financially  supported  by  the 
Universite  Paris  7,  one  of  the  organizers  of  the  meeting,  and  by  the  Association 
Frangalse  pour  la  Cybernetique  Economique  et  Technique  (AFCET),  the  Centre 
National  de  la  Recherche  Sclentiflque  (CNRS),  the  European  Research  Office 
United  States  Army,  the  Mlnistere  des  Affaires  EtrangSres,  the  Mlnistere  de 
I'Educatlon  Rationale,  the  Mlnistdre  de  la  Recherche  et  de  I'Enselgnement 
Superleur,  the  United  Nations  Educational  Scientific  and  Cultural  Organization 
(UNESCO).  Additional  fellowships  to  the  participants  were  provided  by  various 
organizations  listed  separately,  to  whom,  as  well  as  to  the  above  mentioned 
organizations,  we  express  our  gratitude. 


Austin  BLA()UIERE  (Chairman) 
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INTRODUCTION 

The  prototype  for  the  class  of  systems  considered  In  this  chapter  is  depicted  1 
Figure  1  and  consists  of  a  dynamical  process  P  (imperfectly  known)  controlled  by  a 
(judiciously  designed)  feedback  law  (operator  F)  acting  on  state  data  generated  by 
sensor  S  and  implemented  via  actuator  A. 


ACTUATOR  A 


UNCERTAIN  PROCESS  P 


SENSOR  S 


FEEnBACK  OPERATOR  F 

0- 

Figure  1.  Prototype  System 


We  assume  (realistically)  that  the  sensor  and  actuator  are  dynamic  elements  of 
the  feedback  loop;  furthermore,  we  adopt  the  viewpoint  that  these  dynamics  are 
“fast"  relative  to  those  of  the  process  P  to  be  controlled.  If  this  is  not  the 
case,  then,  at  the  modelling  stage,  the  sensor  and  actuator  should  be  explicitly 
incorporated  as  an  integral  part  of  the  process  to  be  controlled. 

We  recognize,  of  course,  that  in  the  context  of  nonlinear  systems,  the  concept 
of  "fas. ness"  is  difficult  to  quantify.  Here  we  use  the  term  loosely  to  indicate 
that  the  overall  system  exhibits  a  "two  time  scale"  structure  as  described  in  the 
next  section. 

THE  FULL-ORDER  SYSTEM 

The  above  prototype  typifies  a  general  class  of  singularly  perturbed  uncertain 
systems  which  can  be  decomposed,  by  means  of  a  scalar  parameter  u,  into  two  coupled 


1  Based  on  research  supported  by  the  NSF  and  AFOSR.  This  paper  deals  with  a  special 
case  of  the  problem  considered  In  [8]  and  [9]. 
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subsystems  which  henceforth  will  be  referred  to  as  the  "slow"  subsystem  (with  state 
x(t))  and  the  “fast"  subsystem  (with  state  y(t)).  The  parameter  y,  henceforth 
referred  to  as  the  singular  perturbation  parameter,  can  be  Interpreted  as  some 
measure  of  the  ratio  of  characteristic  times  of  the  fast  and  slow  subsystems. 

We  model  this  general  class  of  systems  by  the  following  coupled  pair  of  dif¬ 


ferential  equations. 

x(t)  •  X(t,x(t),y(t’),u(t)),  x(t)eR".  u(t)eR'”  (la) 

uy(t)  -  Y(t,x(t).y(t),u(t),u).  y(t)eRP,  ue(0,-)  (lb) 

with  measured  output 

z(t)  »  Sx(t)  +  Ty(t).  z(t)€R'’  (ic) 

where  X  and  Y  are  uncertain  functions  with  the  following  structure; 

X(t,x,y,u)  =  Ajjx  +  Ajjy  +  BjU  +  gj(t,x,y,u)  (2a) 

Y(t,x,y,u,u)  =  C(t)CA2^x  +  y  +  B2U]  +  g2(t,x,y,u,u)  .  (2b) 


A^j,  ,  S  and  T  are  known  constant  real  matrices;  C  is  an  uncertain  measurable 
matrix-valued  function;  g^  and  g2  are  uncertain  Caratheodory  functions  (i.e. 
measurable  in  their  first  argument,  continuous  in  their  other  arguments  and 
integrably  bounded  on  compact  sets). 

Note  that  we  require  that  the  dimension  of  the  output  space  coincides  with  the 
dimension  of  the  slow  subsystem  state  space.  We  refer  to  system  (l)-(2)  as  the 
full -order  system  (a  dynamical  system  on 

Now  suppose  that  the  dynamics  of  the  fast  subsystem  are  neglected,  i.e.  suppose 
that  u  is  set  to  zero,  in  which  case  (lb)  reduces  to  an  algebraic  constraint  on 
(la).  This  procedure  yields  the  reduced-order  system  (a  dynamical  system  on  R*^). 
Suppose  further  that  a  feedback  strategy  is  designed  which  guarantees  some  stability 
property  P  for  the  uncertain  reduced-order  system.  (One  such  design  is  proposed  in 
55  and  analysed  in  §6,  using  the  deterministic  framework  developed  in  e.g.  [1-7]). 
Then  the  essential  question  to  be  addressed  is  that  of  structural  stability  of  pro¬ 
perty  P  with  respect  to  singular  perturbation,  i.e.  does  property  P  persist  when  the 
fast  dynamics  are  re-introduced?  More  usefully,  does  there  exist  a  calculable 

threshold  value  y*  >  0  such  that  property  P -persists  for  all  values  of  the  singular 

* 

perturbation  parameter  in  the  interval  (0,u  )? 

Our  objective  is  to  answer  such  questions  affirmatively,  under  additional 
hypotheses  on  the  full-order  system.  The  first  of  these  is  an  assumption  which 
ensures  that  a  well-defined  reduced  order  system  results  from  setting  y  ■  0  in  (lb). 
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Assumption  A1 

(i)  C(  •)  “  Cg  +  AC(  •) .  where  e  Is  known  with  spectrum  o(C^}  C  t'  (the 
open  left  half  complex  plane)  and  aC:  R  *  is  an  unknown  measurable  function 
with  known  bound  <  (sufficiently  small),  viz.  for  all  t,iAC(t)i  <  «.  < 

.  C  ^  T 

I/21P1*  ,  where  P  >  0  (symmetric)  solves  the  Lyapunov  equation  PC^^  +  C^P  +  I  «  0; 

(ii)  g2(-.'.’.-.0)  •  0. 

THE  REDUCED-ORDER  SYSTEM 

Solving  the  algebraic  equation  Y(t,x,y,u,0)  =  0  for  y  (uniquely  in  view  of 


Assumption  Al)  determines  the  function 

(x,u)  H(x,u)  ^  -  [*21*  *  ^2*^^  • 

The  reduced-order  system  associated  with  (1)  is  now  defined  as 

x(t)  »  X^(t,x(t),u(t)),  x(t)€R"  (4a) 

with  output 

2(t)  »  Sx(t)  +  TH(x(t),u(t)),  2(t)€R'’  (4b) 

where 

Y^(t,x,u)  ^  X(t,x,H(x,u)  ,u)  =  7x  +  ¥u  +  'g(t,x,u)  (5a) 

and 

A,j  -  *12*21’  ^  ^  Bj  -  *12^2*  ?(t,x,u)  ^  gj(t,x,H(x,u),u)  .  (5b) 


At  this  stage,  we  loosely  define  our  preliminary  goal  as  that  of  rendering,  by 
feedback,  some  acceptably  small  compact  neighborhood  of  the  zero  state  of  (4)  glo¬ 
bally  attractive.  Thus,  it  is  not  unreasonable  to  require  the  following  of  the 
nominal  linear  system  pair  (A,B): 

Assumption  A2 

(1)  (?r,ff)  is  a  stabilizable  pair, 

(ii)  S  -  TA2J  is  non-singular. 

Now,  let  (S.Yq)  x  R*  (R^  ^  C0.»))  be  a  pair  of  design  parameters  with 

the  properties  (i)'J  is  symmetric  and  positive  definite  (ii)  tq  >  0  if  a(J)^  t~  . 
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These  properties.  In  conjunction  with  A2,  ensure  that  the  Riccati  equation 

lOT  ♦  ♦  a  -  -  0  (6) 

admits  a  unique  real  positive-definite  symmetric  solution  K  >  0.  Hence,  for 
example,  in  the  absence  of  uncertainty  (^  =  0)  and  if  S  •  I  and  T  ■  0,  the  output 

■  T 

feedback  law  u  ■  -  YqB  Kz  renders  the  zero  state  of  (4)  asympstotically  stable. 

We  now  impose  some  additional  structure  and  hounds  on  the  system  uncertainty. 


Assumption  A3 


There  exist  known  non-negative  real  numbers  c^,  C2,  Cj,  and  unknown  Caratheodory 
function  e:  R  x  r"  x  R'"  ♦  R*"  such  that; 


(i)-  ■g  =  le; 


and,  for  all  (t,x,u)  6  RxR^xR’"'  , 

(11)  ie(t,x,u)i  <  c^  +  C2  ixi  +  C3  lui  . 

In  the  familiar  terminology,  the  uncertainty  is  assumed  to  be  matched  and  cone- 
bounded.  The  more  general  case  of  unmatched  and  non-conebounded  uncertainty  is  con¬ 
sidered  in  [8]  and  [9],  albeit  at  the  expense  of  a  considerably  more  complicated 
controller  design. 


Define  A;  P  r"”"  and  Fj,  r2  R  as  follows: 

Alif)  »  ■  tB2B^K 

rCl.  -);  C2  •  0 

r.  ;  Y  ^  (1  -  c,)*l[Tn  ♦  C2)ia’^ll] 

Mfl.  -):  Cj  >  0  -  ^  ^ 

r2  My:  I  S  -  TA(y)  I  *  0;  ic(y)  <  (1  -  Zk^iPi  )/ZlPCol  +  Zk^'P')) 

where 

k(y)  ^  yiB2B^K[S  -  TA(y)]'^T1I  . 

Then  the  following  additional  assumption  is  required. 

Assumption  A4 

r*  n  Tj  *Ji. 


(78) 

(7b) 

(7c) 


(7d) 
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PROBLEM  FORMULATION 

Suppose  a  (time-dependent)  output  feedback  control  function  (t.z)  ►  q(t,z)  is 
designed  which  guarantees  that  the  feedback-controlled  reduced-order  system 
(viz.  u(t)  «  -  q(t,2(t))  in  (4))  possesses  some  desired  stability  property  P  ,  then 
the  basic  question  to  be  addressed  is  that  of  robustness  of  P  with  respect  to  singu¬ 
lar  perturbation,  where  the  singularly  perturbed  system  is  defined  by  (1)  with 
u{t)  «  -  q(t,z(t));  in  particular,  does  there  exist  a  (calculable)  constant  u*  >  0 

such  that  the  full  system  (1),  under  output  feedback  control  u(t)  •  -  q(t,z(t)), 

* 

possesses  property  P  for  all  values  u€(0,ii  )? 

Here,  we  take  the  desired  property  P  to  be  the  existence  of  a  compact  set  [  c:  r'’ 
(respectively  I  c  fi"‘''P)  containing  the  origin  which  is  a  global  uniform  attractor 
for  the  reduced-order  system  (respectively,  the  full-order  system)  in  the  following 
sense. 


Definition  1 

A  compact  set  51  c  is  a  global  uniform  attractor  for  the  system 

w(t)  -  5(t,w(t)),  w(t)  €  r'’  (*) 

if  the  following  properties  hold: 

(i)  Existence  and  continuation  of  solutions:  For  each  pair  (tp,w°)ER  x  R*^ 
there  exists  a  solution  w:  [t^,tj)  ♦  R*^  (absolutely  continuous  function  satis¬ 
fying  (*)  almost  everywhere)  with  w(tjj)  »  w°  and  every  such  solution  can  be 
extended  into  a  solution  on  [t^j,-); 

(ii)  Uniform  boundedness  of  solutions:  For  each  r  >  0  there  exists  R( r)  >  0 
such  that  iw(t)i  <  R(r)  for  all  t  on  every  solution  w:  Ctp.")  ♦  R*^  of  (*)  with 

lw(tjj)i  <  r,  where  tg®  R  is  arbitrary; 

(iii)  Uniform  stability  of  For  each  d  >  0  there  exists  D(d)  >  0  such  that 
w(t)ej  +  dB  for  all  t  on  every  solution  w;  [t^,*)  ♦  R*'  of  (*)  with 
w(tjj)eJ  ♦  0(d)8  where  t^^  is  arbitrary  (note,  8  denotes  the  open  unit  ball  in 

R*'  and,  for  4  >  0,  J  +  SB  denotes  the  set  (o  ^  p:  o  e  ipi  <  5}); 


8 


^  ^  ■  ■  ■  1^  “sA 


« 


(iv)  Global  uniform  attractivity  of  £:  For  each  d  >  0  and  r  >  0  there  exists 
t{d,r)  >  0  such  that  w(t)ej  +  r8  For  all  t  >  t^^  +  T(d,r)  on  every  solution 

w:  [Fq.")  *  R**  of  (*)  with  w(tQ)€j  ♦  d  8  ,  where  t^^e  R  Is  arbitrary. 

In  the  next  section,  we  construct  a  feedback  strategy  which  ensures  property  ? 
for  the  reduced-order  system  (4). 


NONLINEAR  OUTPUT  FEEDBACK 

Choose  e^,  t2  >  0;  these  are  design  parameters  and  can  be  chosen  arbitrarily 
small.  Define  p;  RxP^  •»  R*”  as 


p(t,x)  4  Pq{x)  +  pj(x)  . 

The  function  p^  is  linear  and  is  given  by 

p^(x)  ft  YjB^x 

where  YjeR"*^  satisfies 
Y^€r*  . 

The  function  p^  Is  nonlinear  and  bounded  and  Is  given  by 


Pl(x)  ft 


I  Pl4l(0l  ^ Kx) 


where  p^eR  satisfies 


If  T  -  0  or  82 
otherwl se 


0 


(8a) 


(8b) 


(8c) 


(8d) 


Pj  >  (1  -  C3)Cj  (8e) 

and  tj:  r"  ♦  R*"  Is  any  smooth  (cM  function  which  satisfies 

I4^(v)i  <  1  ,  <v,4j(v)>  >  ivi  -  Vv€R’’’  (8f) 

and  which  has  bounded  derivative  D^j;  I.e.,  there  exists  R*  such  that 
iO*j(v)i  <  for  all  veR"*.  The  proposed  output  feedback  control  function 

q:  RxR"  •»  R"*  Is  now  defined  by 

q(t.z)  ft  p(t,  [S-TA(yj)]'^z)  .  (9) 

Loosely  speaking,  the  linear  component  (8b)  of  the  control  stabilizes  (If 
necessary)  the  nominal  linear  system  and  counteracts  part  of  the  uncertainty  e  while 
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nonlinear  component  {8d)  (when  active)  counteracts  the  remaining  part  of  e. 

As  an  example  of  a  function  satisfying  the  above  requirements,  consider  the 
function 


♦  v  ivl  +  e^]'*u 


for  which  (8f)  clearly  holds,  and  moreover,  is  with  iD^i(v)i  <  Cj'l  for  an 


V  eR*^. 


A  COMPACT  ATTRACTOR  FOR  THE  OUTPUT  FEED8ACK  CONTROLLED  REDUCED-ORDER  SYSTEM 

For  the  reduced-order  system  (4),  it  may  be  verified  that  q(t,z(t))  ■  p(t,x(t)). 
Hence,  setting  u(t)  •  -  q(t,2(t))  in  (4a)  yields  the  system 

*x(t)  -  F^(t,x(t)),  x(t)  S  r''  (10a) 


Fp(t.x)  »  Kx  •  ■ffp(t,x)  ♦■g(t,x,  -  p(t,x)). 

As  shown  in  [9],  system  (10)  possesses  stability  property  P  . 

To  this  end,  we  define  V:  r"  ♦  R*  (a  Lyapunov  function  candidate)  by 


V(x)  4  <x.Kx>  .  (11 

Theorem  1, 

There  exists  a  closed  ellipsoid 
ft  (xeR":  V(x)  «  r\}  . 

0 

where  r^^  is  defined  in  [9],  which  is  a  global  uniform  attractor  for  system  (10). 

Our  next  objective  is  to  show  that  property  P  is  not  destroyed  by  the  re- 
introduction  of  the  fast  dynamics. 


A  COMPACT  ATTRACTOR  FOR  THE  OUTPUT  FEEDBACK  CONTROLLED  FULL-ORDER  SYSTEM 


Define 


h(x)  ft  H(x,  -  p(t.x))  -  -  A(yj)x  ♦  B2Pi(*). 
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Our  final  assumption  is  now  made. 

Assumption  A5 

( i )  For  all  (t,x) , 

igi(t.x.yj,-q(t.Sx+Tyj))  -  g^(t.x.y2,-q(t.Sx+Ty2))i  <  Xiyj-y2i  Vyi,y2 

where  X  >  0  is  a  known  constant; 

(ii)  for  all  (t.x.y)  and  u  >  0  . 

ig2(t.x,y.-q(t.Sx-t-Ty),u)l  <  u[xily-h(x)i  +  *  Xj] 

where  <2*  •'3  *  0  are  known  constants. 

While  Assumptions  1  to  5  might  appear  somewhat  esoteric,  it  is  stressed  that  the 
class  of  systems  which  satisfy  these  hypotheses  is  far  from  trivial;  for  example, 
the  assumptions  hold  for  a  class  of  uncertain  systems  with  parasitic  actuator  ano 
sensor  dynamics  considered  in  [lOl. 

Let  functions  F:  PxR%RP  ♦  R"  and  G;  RxR'’xRPxR*  *  RP  be  given  by 

F(t.x,y)  ^  A^jX  ♦  A^2>'  *  B^qft.Sx+Ty)  +  gi(t,x,y,.q(t,Sx+Ty>)  (13) 

-  F^(t,x)  +  A^2CyP(*^3  *  BjCp(t.x)-q(t,Sxt-Ty)] 

+  gi(t,x.y.-q(t.Sx+Ty))  -  gj(t.x.h(x)  ,-p(t,x)) 

G(t,x.y.u)  ^  C(t)CA2jX  +  y  -  Bjqlt.Sx+Ty)]  ♦  g2(t  ,x  ,y .-q( t ,Sx*Ty ) ,u  )  (14) 

•  C(t)[y-h(x)]  +  C(t)B2Cp(t,x)-q(t,Sx+Ty)]  +  g2(t.x,y,-q(t,Sx+Ty) ,u) . 

Then  the  problem  under  consideration  reduces  to  that  of  determining  a  threshold 
value  u*  >  0  (if  such  exists)  such  that  the  system  (two  coupled  subsystems): 

x(t)  -  F(t.x(t),y(t))  (^5*' 

wy(t)  -  G(t,x(t),y(t),g) 

possesses  stability  property  P  for  all  u€{0,u  )•  resolve  this  question  via  an 
analysis  akin  to  that  of  [11]. 

As  stated  in  [8]  and  shown  in  [9],  the  following  theorem  establishes  property? 
for  the  full  order  system  under  output  feedback  control. 

Theorem  2. 

*  >  0  such  that,  for  all  u€(0,u*),  a  certain  ellipsoid  is  a 


There  exists  a  w 


global  uniform  attractor  for  system  (15);  the  value  of  u  and  the  definition  of  the 
attracting  ellipsoid  are  given  in  [8]  and  [9].  Moreover,  the  reduced  order  dynami¬ 
cal  behavior  is  recovered  as  u  *  0.2 


EXAMPLE:  UNCERTAIN  SYSTEM  WITH  ACTUATOR  AND  SENSOR  DYNAMICS 

Consider  the  uncertain  system 

>:(t)  •  Ax(t)  +  [B  +  AB(t)]yj(t)  +  d(t,x(t)),  x{t)  €  r" 
with  actuator  dynamics 

uyj(t)  *  [Cj  +  ACj(t)](yj(t)  -  u(t)),  y^{t),  u(t)  e  r"’ 
and  sensor  dynamics 

uy2(t)  =  [^2  iiC2(t)](y2(t)  -  x{t)),  y2(t)  e  r" 

where  the  known  nominal  system  matrices  A,  B,  Cj ,  C2  satisfy  the  following 

HI 

(i)  (A,B)  is  a  stabilizable  pair; 

(ii)  o(C^)  c  t"; 

(iii)  0(02)  c  C** 

The  uncertain  functions  aB(-)  and  d(",")  are  assumed  to  satisfy 
H2 

(i)  aR(*)  •  RE('),  where  E{  ^(unknown)  is  measurable  with  iE(t)i  <  S  <  1  Vt; 

(ii)  d(-,-)  =  where  g(-,-)  is  a  Caratheodory  function  with 

lig(t,x)ll  <  a-illxll  +  02  V(t,x)  and  where  ay  02,6  are  known 

constants. 

Let  P  (symmetric  and  positive  definite)  denote  the  unique  solution  of 


^Loosely  speaking,  in  the  sense  that  the  projection  of  the  attracting  ellipsoid 
onto  r”  approaches  the  attracting  ellipsoid  of  reduced  order  system. 


(16a) 

{16b) 

{16c) 
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Then  the  uncertain  functions  AC^(*)  and  aCgC*)  are  assumed  to  satisfy 
H3 

id1ag{aC^(t),  aC2(t)}i  <  <  l/2iPi  ^  Vt,  where  is  a  known  constant. 


The  above  can  be  interpreted  in  the  context  of  system  (l)-(2)  by  making  the 
following  identifications: 


e  R**,  p  ^  m  +  n 


•  A,  Aj^2  *  [B  •  0]  t  ^21 


Bi  *  0  9  Bn 


(18a) 


{18b) 


.  S  •  0.  T  =  [0  :  I]  (18c) 

C(t)  .  Cg+AC{t)  ,  Cg  -  diag{C^,  C2},  AC(t)  =  diag{4Cj(t),  4C2(t))  (18d) 

g|(t,x,y,u)  «  d(t,x)  +  BE(t)Cl  ♦  0]y  (18e) 


g,  5  0 


(18f) 


In  view  of  Hl(ii),(iii)  and  H3,  it  is  clear  that  Assumption  A1  holds  for  this 
system. 


Now, 

A  •  Ajj  -  1^12^21  *  ^1  *  * 

(19a) 

R  ■  “  ^12^2  *  ”  ^12^2  *  ^ 

(19b) 

and  hence,  in  view  of  Hl(1),  it  follows  that  Assumption  A2  holds. 


Also, 

H(x,u)  ■  -  i^2i*  *  ■  [xj 

(20) 

and 

g(t,x,u)  •  gj{t,x,H(t,x)  ,u)  ■  Be(t,x,u) 

(21a) 

where 

(2lb) 

e(t,x,u)  -  g(t,x)  >  E(t)u  . 

13 


Thus,  in  view  of  H2,  it  is  clear  that  Assumption  A3  holds  with  Cj  ■  g. 
Proceeding, 

A(t)  - 

S  -  TA(y)  ■  I,  •c(y)  «  riB^Ki 

Tj  *  (1-2k^|P|){2iP  C^I  +  2ic^iPi)"^ib\i*^)  C  R  . 


{22a) 

(22b) 

{22c) 


Assumption  A4  now  reduces  to  the  following: 
A4*:  T  <  (1-2'Cj.iPi)(1+2k^iPi)“^iB^K|‘^  . 


Finally,  it  is  readily  verified  that  Assumption  A5(ii)  holds  trivially  (since 
g2  s  0)  and  A5(i)  holds  with  X  •  giBi. 

A  specific  example  of  this  subclass  of  systems  is  considered  in  detail  in  [9], 


OTHER  METHODS 

An  approach,  differing  from  the  one  proposed  here,  can  be  found  in  [12-15].  In 
these  references,  the  design  procedure  requires  the  sequential  construction  of 
controllers  which  assure  existence  of  global  uniform  attractors  for  (i)  an  approxi¬ 
mation  of  the  reduced  order  ("slow")  subsystem,  and  (ii)  the  "fast"  subsystem  under 
the  influence  of  the  slow  uncertainties.  The  controller  for  the  full  syst-m  is  then 
obtained  as  the  sum  of  these  subsystem  controllers. 
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Rdsumy 


II  est  connu  que  toute  solution  rgguliyre  d'une  Equation  d* Hamit  ton- Jacobi -Bellman 
assoclde  i  un  probtdme  de  contrdte  optimal  peut  #tre  utltlsSe  pour  la  verification  de 
1 'optimality  d'une  trajectoire  du  systyme,  alnsi  que  pour  la  construction  des 
retroactions  optlmates. 

En  general  de  teltes  solutions  regutieres  n'existent  pas  et  on  Introdult  les 
solutions  general Isees  (solutions  de  viscosite  ou  autres). 

Dans  cet  expose  on  pose  la  question  sulvante  :  quel les  sont  des  conditions 
necessalres  et  suffisantes  pour  qu'une  fonctlon  V  :  R  x  r"— ^  Ru[“  ‘"jverlfie 
les  proprietes  sulvantes  ; 

1)  V  est  monotone  1e  long  des  trajectolres  du  systeme 

2)  V  est  constante  le  long  d'au  molns  une  trajectoire  (qui  est  une  solution 

optimale  du  probieme). 

Les  proprietes  1)  et  2)  sont  cruclales  pour  1 'application  des  techniques  de 
verification. 

On  demontre  aussi  que  de  telles  functions  forment  une  sous-classe  des  solutions 
de  viscosity  de  I'equatlon  d'Hamllton-JacobI -Bell man. 

Les  proprietes  1)  et  2)  sont  tou jours  verl flees  par  la  fonctlon  valeur  associee 
au  probietne.  Nals  cette  derniere  etant  souvent  discontinue,  11  est  Interessant  de 
trouver  d'autres  fonctlons  qui  satisfassent  1)  et  2). 

La  deuxieme  partle  de  1 'expose  concerne  la  construction  des  retroactions 
optlmales  associees  i  de  telles  fonctlons. 
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ABSTRACT 


We  establish  a  general  robust  control  result  for  linear  time-invariant  uncertain  systems  using  the  Lyapunov  approach 
initiated  by  Leitmann  and  Gutman.  We  show  that  systems  satisfying  matching  conditions  are  handled  by  this  result. 
We  give  necessary  and  sufficient  conditions  for  the  existence  of  a  robust  sliding  mode  controller.  We  show  that  its  ex¬ 
istence  implies  the  existence  of  a  robust  linear  controller.  A  counter  example  is  provided  to  establish  that  the  converse 
does  not  hold.  The  feedback  controllers  treated  are  functions  of  the  complete  state  without  any  dynamic  compensation, 

I.  rSTRODLCTION 

The  Lyapunov  approach  to  uncertain  systems  received  an  initial  thrust  by  Leitmann  and  Gutman,  1 1)  - 17).  for  systems 
satisfying  matcliing  conditions.  They  are  joined  by  numerous  authors  (  e.g.  |8|  -  (33])  in  extending  the  Lyapunov  ap¬ 
proach  to  handle  more  general  systems  since  it  is  well  suited  for  addressing  structured  uncertainty.  Our  work  herein  fo¬ 
cuses  on  applying  the  Lyapunov  approach  to  systems  which  have  constant  uncertainties  but  do  not  necessarily  satisfy 
the  matching  conditions.  It  builds  on  the  work  of  [9|,  |I4|,  and  |20|  - 133|.  Our  main  objective  is  to  establish  a  robust 
control  result  based  on  the  Lyapunov  approach  which  generalizes  some  of  the  past  work  on  linear  uncertain  systems 
with  constant  uncertainties.  We  specifically  consider  linear  and  sliding  mode  controUen  and  give  necessary  and  sufficient 
conditions  for  their  existence.  We  prove  that  the  existence  of  a  robust  stabilizing  sliding  mode  controller  implies  the 
existence  of  a  robust  stabilizing  linear  controller.  The  converse  docs  not  hold.  We  provide  a  counter  example  showing 
the  existence  of  a  robust  linear  controller  in  the  absence  of  such  a  sliding  mode  controller.  Herein,  we  use  the  term  sta¬ 
bility  to  mean  that  the  poles  are  in  the  left-half  plane,  i.e.,  asymptotic  stability  or,  equivalently,  that  the  characteristic 
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polynomial  is  Hufwitz.  We  say  that  a  controller  is  tobuA  if  h  asymptotically  stabilizes  the  system  for  all  uncertainties. 
We  treat  both  the  scalar  input  and  the  multi-input  problems. 

We  investigate  the  robust  control  of  linear  time-invariant  uncertain  systems  that  ate  not  required  necessarily  to  satisfied 
matching  conditions; 


X  -  A{y)x  B{y)u,  y  tV  (1) 

where  ii(y)  is  a  run  uncertain  matrix,  B(y)  is  an  nxm  uncertain  matrix  with  full  rank  (m  £  n)  and  y  belongs  to  a  set 
of  uncertainties  F  where  F  is  a  simply  connected,  compact  subset  of  p-dimensional  Euclidean  space  S'.  We  assume  that 
A(y)  and  B(y)  are  continuous  with  respect  to  the  uncertainty  argument  y  e  F  .  In  this  paper  we  consider  only  fuU  state 
feedback  controllers  u(x),  i.e.,  those  which  are  functions  of  the  state  x  only.  That  is,  we  do  not  address  dynamic  com¬ 
pensation  as  part  of  the  feedback  controller.  We  require  that  system  (1)  satisfy  the  controllability  assumption: 

ASSUMPTION  I.  For  each  y  «  F  the  pair  (A(y),  B(y))  is  controllable. 

The  controllability  assumption  is  equivalent  to  the  assumption  that  closed-loop  poles  can  be  arbitrarily  placed  by  a 
suitable  gain  matrix.  We  state  this  equivalent  assumption: 

ASSUMPTION  r.  For  each  y  e  F  and  prescribed  eigenvalues  A(y)  -  (/l|(y) . ‘l,(y))  in  which  imaginary  eigenvalues 

occur  in  complex  conjugate  pairs  there  exists  a  real  gain  matrix  K{y)  such  that  the  closed-loop  matrix 


Am  -  Am  -  Bmxm 


(2) 


has  the  prescribed  eigenvalues  A(y). 

For  arbitrarily  prescribed  eigenvalues  A(y),  y  c  F,  we  can  rewrite  (1)  as 


X  m  A{y)x  +  B(y)£K(y)x  +  u] 


where  Xm  is  the  corresponding  gain  matrix  and  Afy)  satisfies  (2). 


(3) 


The  next  assumption  makes  it  possible  to  define  a  control  law  with  which  to  stabilize  (1)  in  the  presence  of  uncertainties 

y  e  r. 


ASSUMPTION  II.  For  each  y  c  F  there  exist  an  nun  gain  matrix  K'y).  s.n  invertible  mxm  matrix  Rly)  and  an  nxn 
symmetric,  positive  definite  matrix  ^y)  such  that 

O')  (y)  =*  .ilfy)  -  B{y)  K(y)  is  asymptotically  stable 

(ii)  F  —  y?*‘(y)  F^y)  P(y)  is  a  constant  mxn  matrix  where  P{y)  is  the 
symmetric,  positive  definite  solution  of  Lyapunov  equation 


P{y)A{y)  +  A^{y)P{y)  +  e(y)  =  0 

We  make  the  following  assumptiot)  on  the  mxm  matrix  R(y)  which  is  defined  in  Assumption  11. 
ASSUMPTION  III.  For  y  t  F  the  matrix  <l>(y)  defined  as 


0(y) 


R\y)  +  Ptvj 
2 


is  positive  definite  and  has  the  square  root  form 


(4) 


(5«) 


<t>{y)  -  s\)S[y) 

where  5(y)  is  invertible.  The  following  upper  bound  exists  and  is  finite 

h  -  ^[||5-'(y)|l  liS'^fy)  /l*(y)||] 


(5f>) 


(6) 


In  Sections  2*4  and  6  we  show  how  to  use  the  constant  matrix  F  in  establishing  a  robust  controller. 
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2.  MAIN  ROBUST  CONTROL  RESLLT 

Assumptions  I  •  III  pennit  the  development  of  >  robust  control  law  that  is  discontinuous  in  nature.  This  is  established 
in  the  next  theorem. 

THEOREM  1:  If  system  (1)  satisfies  Assumptions  I  •  III  then  the  discontinuous  controller 

pW.  ^  0  (7) 

stabilizes  (1)  for  all  y  e  P  where  p(x)  satisfies 

p(x)  =  Ama^SAfWxIl  (8) 

The  scalar  h  is  given  by  (6)  and  the  gain  matrix  K{y)  is  defined  in  Assumption  11. 

PROOF:  For  y  t  P  let  K{y).  R(y),  Q(y),  P{y)  and  P  be  the  matrices  described  in  Assumption  II.  Define  the  Lyapunov 
function 


P(y)  -  x^Piy)x  (9) 

It  has  the  time  derivative 

P(y)  -  -x^Q{y)x  +  2[s’^(y)P(y)x]''  CA:(y)x  +  u]  (10) 

Using  property  (ii)  of  Assumption  II  this  derivative  becomes 

P(y)  -  -x’'Q(y)x  +  2[Fx]^  R^(y)  C/s:(y)x  +  u(x)]  (11) 

We  show  that  the  control  law  (7)  yields 

P(y)  S  -x^Q{y)x,  y  t  P  (12) 

Since  Q(y)  >  0  (i.e.,  positive  definite)  it  suffices  to  show  that  fV(y)  is  nonpositive; 

W(y)  -  2[Px]^R^(y)[A:(y)x  +  ^x)]  S  0  (13) 


Consider  a  control  law  of  the  form  (7)  in  which  the  scalar  function  p(x)  u  defined  by  (8).  substitution  of  (7)  into  (13) 
yields 


W'fy)  -  2W,(y)  -  2Wj(y)  S  0 

where 


(14) 
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M'.Iy)  -  [/jc3^A^(y)A:(y)* 

(15a) 

(156) 

Eq.  ( ISb)  can  be  rewritten  as 

W'j(y)  -  [/i]''<IKy)-j^p(x) 

(16) 

or,  equivalently  as. 

W'jfy)  -  [S(y)/i]^.^|S-p(x) 

(17) 

where  ^y)  and  S{y)  ate  defined  by  (S)  and  (6).  Making  the  vector  definition 

Eq.  (17)  becomes 

y(y)  -  S(y)Fx 

(18) 

t„ ,  ,  y^(y)y(Y) 

"  lITxil 

(19) 

Eq.  (15a)  can  be  rewritten  as 

« 

where 

f*'i(y)  «  y\y)2(y) 

(20) 

Inequality  (14)  is  met  provided 

Ky)  -  5(y)[«l>(y)r'  R\)K{y)x 

(21) 

H'tCy)  S  H'jiy) 

In  terms  of  (19)  and  (20)  this  inequality  is  given  by 

(22) 

y^(ym  s  />(X),  Fx  #  0 

(23) 

This  inequality  is  met  provided 

IWy)||  S  p(x).  Fx  #  0 

(24) 

Taking  the  norm  of  (21)  yields 

«2<y)n  S  l|5"^(y)«^(r)li  ll/f(yWI 

Multiplying  both  tides  by  the  norm  llS-'(y)il  gives 

(25) 

Observe  that 

U5-'(y)||  Ilz(y)||  s  p(x) 

(26) 

llfjcll  -  ||5-'(y)5(y)/>||  S  B5-'(y)ll  lly(y)|| 

(27) 

/ 
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iioiii  which  it  follows  that 


Multiplying  both  sides  by  p(x)  yields 


,  ^  |5-'(y)ill|y(a  ^  0 


(28) 


p(Jf)  S 


y.r'(y)ii  HKy)ii 
IF^ 


/>(■*). 


fx  #  0 


(29) 


The  inequalities  (26)  and  (29)  yield 


IWy)ll  5 


JKyM 


p(x). 


Fx  ^  0 


(30) 


This  verifies  (24)  which  establishes  (12).  By  the  theory  of  Lyapunov,  the  control  law  (7)  sUbilizes  (1)  for  each  uncer¬ 
tainty  y  c  r. 

3.  ROBUST  CONTROL  IN  THE  PRESENCE  OF  MATCHING  CONDITIONS 

Systems  which  satisfy  the  matching  conditions  of  linear  uncertain  systems.  12)  -  (7),  satisfy  Assumptions  1  •  111.  This 
result  is  given  by  the  next  theorem. 

THEOREM  2:  Let  system  (1)  satisfy  the  Mowing  matching  conditions:  There  exist  an  nxn  matrix  A  and  an  nxm 
matrix  B  and  for  each  y  e  P  there  exist  an  mxn  gain  matrix  D{y)  and  an  invertible  mxm  matrix  n(y)  such  that 


(a)  A(y)  =  A. -t-  50(y). 

(b)  5(y)  -  5n(v). 

(c)  (A,  B)  is  a  controllable  pair 

(d)  ^(y)  is  an  mxm  positive  definite  matrix  where 


Ofy) 


n^(y)  +  n(y) 
2 


(31) 


Then  Assumptions  I  •  Ill  ate  met.  As  a  consequence  of  Theorem  1,  there  exists  a  robust  stabilizing  control  law  of  the 
form 


u 


-Kx  + 


Fx 


P(x) 


(32) 


such  that 
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is  asymptotically  stable  and  such  that 


A  m  A-BK 


F  -  B'^P 

where  Pis  the  symmetric,  positive  definite  solution  of  the  Lyapunov  equation 

pa+Vp  +  q  -  0 

in  which  £  >  0  is  arbitrarily  chosen. 


(33) 

(34) 

(35) 


PROOF:  Conditions  (a)  -  (c)  imply  that  (A(y),  B(y))  is  controllable  for  y  t  T.  Controllability  is  invariant  under  linear 
feedback  and  coordinate  transformation  on  the  input,  (34|.  Thus  Assumption  I  is  met.  Since  (A,  B)  is  controllable  there 
exists  a  gain  matrix  K  such  that  A  of  (33)  is  asymptotically  stable.  Define  the  uncertain  gain  matrix 


K(y)  -  n"'(y)  [D(y)  +  X]  (36) 

Using  conditions  (a)  and  (b)  we  find  A  (y)  of  condition  (/)  of  Assumption  11  reduces  to 

A(y)  ~  A  -  BK  (37) 

and  is,  therefore,  asymptotically  stable  for  y  e  T.  Select  any  ^  >  0.  Let  P  be  the  solution  of  (36)  and  let  F  be  defined 
by  (34).  For  y  t  P  define 


%)  -  n^(y)  (38) 

The  matrix  F  of  condition  (ii)  of  Assumption  II  and  that  of  (34)  are  identical.  That  is,  (34)  can  be  rewritten  as 

F  -  n-^(y)[Pn(y)]''P  (39) 

which,  in  view  of  condition  (b)  and  (38).  is  equivalent  to 

F  -  R-'(y)P^(y)P  (40) 

Thus,  condition  (tr)  of  Assumption  II  is  met  with 

P(y)  ■  P  (41) 

Condition  (d)  implies  Assumption  III  since  B(y)  is  continuous  and  P  is  compact.  That  is,  h  exists  and  is  finite.  Since 
all  condirions  of  Theorem  1  are  met.  the  existence  of  the  stabilizing  control  law  (32)  follows  with 


where  .^(y)  is  defined  by  (36). 


p{x)  m  h  mip|tAr(y)x|| 


(42) 
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4.  ROBUST  CONTROL  IN  THE  ABSENCE  OF  MATCHING  CONDITIONS:  SCALAR  INTLT 

We  show  that  the  robust  control  assumptions  presented  in  |29]  for  scalar  control  satisfy  the  assumptions  of  Theorem  1. 

Consider  system  (1)  with  scalar  control.  The  input  matrix  B(y)  is  a  column  vector.  The  work  in  [29|  assumes  that  the 
system  (1)  is  controllable.  Assumption  1.  Under  this  assumption  there  is  a  unique  coordinate  transformation  T(y) 

z  -  7T(y)x  (43) 

of  (1)  to  the  following  controllable  companion  form,  |34|, 

Z  -  +  a,a(x)  (44) 

where 


AMy)) 


1  0 
0  1 
0  0 


0  0 
0  0 
1  0 


(45n) 


-<j(o{y))  »  T(y)A(y)7~'(r)  (456) 

and 

5,  -  [1.0,0,...,0]^  (46a) 

S,  -  ny)S(y)  (466) 

The  vector  a(y)  «  (a,(y) . <t,(y))  is  the  coelTtcient  vector  of  the  open-loop  characteristic  polynomial: 

a,(s)  -  det[s/-A(y)]  (47) 

We  need  the  following  definition  in  order  to  introduce  the  next  assumption  of  |29|-[3II. 


DEFINITION  I:  The  row  vector  F,  -  (/*„,  /*,„ ... ,  /’,,)  is  said  to  be  n  —  1  stable  provide  I’ll  ^  0  Mtil  the  polynomial 

Ftii"-'  +  +  ...  +  F,^  ~  0  (48) 

is  Hurwitz  (i.e.,  all  eigenvalues  ate  in  left-half  plane). 

ASSUMPTION  IV:  There  exist  an  uncetain  yo  t  P  and  an  n  —  I  stable  row  vector  P,(y,)  such  that 

f’i(y)  -  f’i(yo)7‘(yo)7-'(y)  (49) 

is  n  —  1  stable  for  all  y  t  P 
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The  concept  of  a  vector  being  n  —  1  stable  is  ftindamental  in  the  asymptotically  stable  solution  of  Lyapunov  equation. 
This  result  is  presented  in  the  next  lemma.  Its  proof  is  given  in  (30|. 

LEMMA  I.  Let  d  a  (o,, ... ,  qj.  Define  A(a)  to  be  in  the  controllable  companion  form  (45).  Let  ?  be  the  solution  to 
the  Lyapunov  equation 


PA{a)  +  A\a)P  +  2-0  (50) 

where  2  >  0  and  Q  -  Q^.  Then  A(a)  is  stable  if,  and  only  if,  f*,  is  a  -  1  stable  where  P,  is  the  first  row  of  P. 

PROOF;  See[30|. 

The  next  lemma  is  a  consequence  of  Lemma  I. 

LEMMA  2.  Suppose  Assumption  IV  holds.  For  each  y  e  F  define  Q{y)  >  0,  2(y)  •  Then  for  each  y,  there  is 

a  unique  stable  coefficient  vector  a(/)  satisfying  Lyapunov  equation 


P{y)A(a{y))  +  A\cHy))P{y)  +  2(y)  -  0  (51) 

where  P,{y),  the  first  row  of  P{y),  is  prescribed  under  Assumption  IV.  That  is,  A(d(y))  is  stable  for  y  t  F. 

The  above  lemmas  are  used  in  (he  next  theorem  to  establish  a  stabilizing  controller  for  system  (I). 

THEORE.M  3.  If  Assumptions  I  and  IV  hold  then  there  is  a  stabilizing  controller  for  system  (1)  having  the  form 


where  f  is  a  constant  row  vector  and  p(x)  is  a  nonnegative  scalar  function  of  the  slate  x. 

PROOF:  Since  system  ( 1)  is  controllable  for  each  uncertainty  ycF  it  can  be  transformed  to  the  controllable  companion 
form  (44).  Assumption  IV'  implies  there  is  a  stable  coefficient  vector  ^y)  for  y  t  F  such  that  (51)  is  satisfied.  Define 
dfy)  to  be  the  difference  between  the  stable  coefficient  vector  a(y)  and  the  open-loop  characteristic  polymonial  coefficient 
vector  a(y)  of  System  (1) 


»(y)  -  a  (y)  -  afy) 


(53) 


Note  that  the  negative  of  a(y)  is  contained  in  the  fust  row  of  (45).  Substitution  of  (53)  into  (44)  yields 

i  -  ^,(3(y))r  +  B,C<T(y)T(y)*  +  u(*)]  (54) 

after  making  use  of  (43).  We  use  the  symmetric,  positive  definite  solution  P{y)  of  (51)  to  construct  the  Lyapunov  func¬ 
tion 


V(y)  -  t^P{y)z 

Taking  its  derivative  gives 

(SS) 

P(v)--z^e{y)z  +  2[Px3^Cir(y)  r(y)x  +  u(x)] 

where  F  satisfies 

(56) 

F  -  />,(yo)r(yo) 

and  as  a  consequence  of  Assumption  IV  we  have 

(57a) 

F  »  Pi{y)T(y) 

or,  equivalently, 

(576) 

F  -  B[P{y)T{y) 

where  P{y)  satisfies  (51)  and  T(y)  satisfies  (43).  Any  admissible  control  law  u(x)  satisfying 

(57c) 

u(x)  s  -  maj5t<r(y)T(y)jc3  ,  Fx  >  0 

(58a) 

u(x)'2  m;pC<r(vlT(y)x]  ,  Fx  <  0 

(586) 

stabilizes  (1)  since  for  such  a  control  law 


l>(y)S-z^(?(y)z.  ytV  (59) 

The  majuma  of  (58)  exist  since  P  is  compact  and  since  the  functions  <7(y)  and  T(y)  are  continuous  on  P.  An  admissible 
control  law  satisfying  (58)  is  (52)  where 


/>(jc)  -  maj5||<T(y)7Xy)*1l 


(60) 


and  F  is  given  by  (57).  In  the  next  theorem  we  esublish  that  a  system  satisfying  Assumption  IV  also  satisfies  Assumption 


THEORE.M  4:  If  the  system  (1)  satisfies  Assumptions  I  and  IV  risen  Assumptions  II  and  III  are  met. 


PROOF:  We  make  the  following  identifications 


A(y)  ■  7~'(y)  A(a(y))  T(y) 

(61a) 

P(y)  -  7'^(y)F(y)T(y) 

(616) 

C(y)-  t\)Q(y)T(y) 

(61c) 

X(y)  -  *(y)  F(y) 

(61d) 

when  T{y)  is  defined  by  (43),  where  P[y),  ^y)andd(^y))  are  defined  by  (51)  and  where  e(y)  is  defined  by  (S3).  The 
matrix  A(a(y))  is  asymptotically  stable.  This  follows  from  Lemma  2  and  the  fact  that  eigenvalues  are  invariant  under 
coordinate  transformation.  From  (44),  (45),  (S3)  and  (54)  it  follows  that 


A{y)  =  A(y)  -  .S(y)^(y)  (62) 

so  that  condition  (i)  of  Assumption  III  is  met.  The  vector  F  of  (57)  satisfies 

F»5^(y)F(y)  (63) 

where  P{y)  is  the  solution  of  the  Lyapunov  equation 

P(y)  A(y)  +  J^(y)  P(y)  +  Q(y)  -  0  (64) 


which  shows  that  condition  (ii)  of  Assumption  II  is  met.  Here,  the  scalar  R  •>  1.  Thus  Assumption  III  is  also  met. 


Theorems  3  and  4  establish  that  Assumption  IV  implies  Assumption  11.  The  converse  need  not  hold.  Thus  Assumption 
IV  is  a  stronger  assumption.  Assumption  IV  admits  a  sliding  mode  controller  (52).  From  the  next  theorem  we^'  that 
it  also  admits  a  stabilizing  linear  controller. 

THEOREM  5;  If  Assumption  I  and  IV  hold  then  there  exists  a  stabilizing  lirtear  control 

um~  cFx  (65) 

where  F  is  defined  as  in  Theorem  3  and  the  scalar  c  satisfies 

e  >  ^mi^||(?"'(y)ll  nMfll/((y)ll*  (66) 

where  Q(y) ,  y  c  F,  is  defined  as  in  Lemma  2  and  where  X(y)  is  given  by  (61d). 


PROOF:  SceI31|. 


The  nuudina  of  (66)  exist  since  K(y)  is  continuous,  P  is  compact  and  the  matrices  Q(j/)  am  chosen  in  a  continuous 
manner.  Usually  Q(y)  is  set  to  be  the  identity  I  or  it  is  computed  from 

e(y)  -  r-^(y)  er-'(y)  (67) 

where  Q  is  a  prescribed  symmetric,  positive  definite  matrix.  The  next  resuh  gives  an  equivalence  between  Assumption 
TV  and  a  minimnm  phase  condition  on  the  system. 

THEOREM  6:  Assumption  TV  is  met  if,  and  only  if,  there  is  a  row  vector  F  such  that 

Flsl~A(y)2-'  S(y).  y  c  P  (68) 

is  minimum  phase  with  n-1  transmission  zeros  where  I  is  the  nxn  identity  matrix.  That  is,  the  determinant 


29 


m 


ny)Lil-A{y)r'B{r)  -  \{s)  ''  ~ 


Multiplying  Ixjih  sides  by  FT->(y)  yields 


nsl-A(y)T'B(y) 


/T~'(y)[s*~*s"~^-JiF  _ 


(75) 


(76) 


.after  making  use  of  (72).  The  transmission  zeros,  C35]  ,  of  (76)  are  the  n-1  staUe  eigenvalues  of  the  (n-1)  staUe  /*,(y) 
tow  vector  of  (70).  This  proves  that  (68)  is  minimum  phase.  From  (76)  we  have 


det[s/-.4(y)]  F[s/-^(y)r'5(v)  -  0 
A  reciprocal  form  of  (77)  is  given  by,  (;34], 


(77) 


det 


sl-A{y)  B(y) 
-F  0 


(78) 


which  yields  (69).  Since  P,{y)  is  n-1  stable  it  follows  that  (71)  is  Hurwitz.  Thus  (69)  is  Hurwitz. 

Conversely,  if  there  exists  an  F  such  that  (69)  is  Hurwitz  then  the  vector  Pfy)  defined  by  (70)  is  n-1  stable  and  As¬ 
sumption  IV  is  met.  From  the  above  theorem  we  have  the  corollary. 

COROLLARY  1.  A  necessary  and  sufficient  conditions  for  the  existence  of  a  stabilizing  sliding  mode  controller 


“--■jgjj-Pf*).  (79) 

of  (1)  is  the  existence  of  a  row-vector  F  such  that  (69)  is  Hurwitz  for  all  ytP. 

The  existence  of  a  stabilizing  linear  controller 

«■  — Xx  (80) 

docs  not  imply  the  existence  of  a  subilizing  sliding  mode  controller  (79).  Bcfim  this  is  iUusiratad  by  an  exam^  we  give 
necessary  and  sufficient  conditions  for  the  existence  of  a  linear  controller  (80). 


THEOREM  7:  A  neceuaiy  and  suflicient  condition  that  there  exist  a  stabilizing  linear  controller  (80)  of  system  (1)  is 
that  there  exists  a  row  vector  K  such  that  the  following  determinant  is  Hurwitz: 


det 


sl-A{y)  B(y) 
-K  1 


0,  y  e  r 


(81) 


PROOF:  Suppose  there  is  a  row  vector  K  such  that  (80)  asymptotically  stabilizes  (1).  The  feedback  matrix 


AM  -  A{y)  -  B{y)K,  y  t  T  (82) 

is  asymptotically  stable  and  the  determinant 

<(i)  -  det[f/-i4,(y)]  =  0  (83) 

is  Hurwitz.  Eq.  (83)  can  be  written  as  the  following  series  of  identities,  [34], 

a^Cr)  -  det{Cj/-^(y)]  [/ +  [r;-^(y)r'%)A:]}  (84<2) 

<tj(r)  =  a,(r)  det[/  +  [r/  -  ^(y)]"'B(y)/i:]  (846) 

<(1)  -  [l+  Klsl-  ^(y)]-'B(y)]  (84c) 


where  a,(r)  is  the  open-loop  characteristic  polynomial  (47).  The  reciprocal  form  of  (81)  is  (84c),  134).  That  is,  (81)  and 
(84c)  are  identities.  Therefore,  (81)  is  Hurwitz  if,  and  only  if,  (83)  is  Hurwitz.  Eq.  (84c)  can  be  used  to  prove  Theorem 
5.  If  (76)  is  Hurwitz  then  with 


A-cf 

Eq.  (84c)  becomes 

-  <^(r)[l  +  cFLsi  -  A(y)r'S{y)] 

which  is  Hurwitz  for  suiEciently  large  c.  That  is,  in  view  of  (71)  -(76),  Eq.  (86)  can  be  rewritten  as 
<(r)  -  j"  -I-  cP,(y)[r-'.j"-’...r  lY  +  [Vrl-r"] 


(85) 

(86) 

(87) 


in  which  the  last  term  is  an  n-l  order  polynomial  that  is  dominated  by  the  middle  tenn  for  large  c.  The  first  two  terms 
give  a  Hurwitz  polynomial  for  sufficiently  large  c.  As  a  consequence,  the  existence  of  a  robust  stabilizing  sliding  mode 
controller  (52)  in^tlies  the  existence  of  a  robust  stabilizing  linear  controller  (65).  In  general,  the  converse  does  not  hold 
as  it  illustrated  by  the  following  example. 
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5.  EXAMPLE  OF  ROBUST  LINEAR  CONTROLLER  WnWOUT  SLIDING  MODE  CONTROLLER 


Consider  the  uncertain  system 


where  T  *  [  —M,  A/]  and  where  M  is  a  positive  scalar  greater  than  1 


The  determinant  of  the  controllability  matrix  CR(y),  A£(y)l  is  given  by  y^  +  1  which  satisfies  the  equality 

y^  +  1  >  0  V  y  t(  — oo,  oo) 


The  system  (88)  is  controllable  for  all  uncertainties  y.  Thus  Assumption  I  is  satisfied.  The  requirement  for  the  existence 
of  a  stable  sliding  mode  surface 


depends  on  (69)  being  Hurwitz.  For  our  example  system  (88)  Eq.  (69)  reduces  to  the  first  order  polynomial 

(/"ly  +  /ilA  +  (y/i-Fii-O 

which  is  Hurwitz  for  y  c  F  provided  the  coefficients  are  positive 

FiV  +  Fi  >  0,  y  e  r 
yFi-Ff  >  0,  y  t  r 

Evaluating  the  first  inequality  at  y  »  1  and  the  second  at  y  »  —  1  pve  the  contradicting  inequalities 

Fj<-F, 


That  is,  there  exists  no  (F,,  F^  utisfying  (69)  for  ysC  —1,13  which  is  a  subset  of  T  .  Consequently,  there  is  no  stable 
sliding  mode  surface  (91)  on  which  a  robust  sliding  mode  controller  (S2)  can  be  designed  for  yc[  -1, 1]. 

The  tequiteroent  for  the  existence  of  a  robust  subiiizing  linear  feedback  controller  (80)  is  that  (81)  is  Hurwitz.  The 
characteristic  polynomial  of  (81)  is  given  by 


/ 
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t-  <l,(y)i  +  02(y)  -  0 

where  robustness  follows  fiom  poshiveness  of  the  coefficients 

(95) 

ai(y)  “  Afj  +  yAT)  >  0,  y  t  T 

(96a) 

OsW  “  yAfj  -  ATi  +  1  >  0,  y  t  r 

The  following  gain  vector  X  -  (X„  Xj)  provides  a  robust  linear  controller  (80) 

(966) 

ie 

1 

o 

(97a) 

-  _  1 
^  M  +  e 

(976) 

where  t  >  0.  Substitution  of  the  gain  vector  (97)  into  (96)  gives 


Xi>  0,  r  t  r 

(98a) 

y  >  ~(M  +  t),  y  t  r 

(986) 

The  inequalities  (96)  are  net.  Thus  (81)  is  Hurwitz  which  implies  that  the  linear  controller  defined  by  (97)  robustly 
stabilizes  (88).  Consequently,  (88)  has  a  robust  stabilizing  linear  controUer  but  no  stabilizing  sliding  mode  controller. 


6.  ROBUST  CONTROL  IN  THE  ABSENCE  OF  MATCHING  CONDIHONS:  Mt'LTI-INTLT 

The  multi-input  case  parallels  that  of  the  scalar  case,  Section  4.  We  consider  a  condition  similar  to  (69)  and  show  that 
it  leads  to  necessary  and  sufficient  conditions  for  the  existence  of  a  sliding  mode  controller  (7).  In  this  section 
B{y),  y  s  r,  is  an  nxm  uncertain  matrix  with  full  rank  {m  £  n)  We  consider  system  (1)  for  which  Assumption  I  bolds. 
Our  main  result  for  a  robust  sliding  mode  controller  u  given  in  the  next  theorem. 


THEOREM  8.  A  robust  stabilizing  sliding  mode  controller  (7)  exists  for  system  (1)  in  which  Assumption  I  holds  if, 
and  only  if,  the  following  determinant  is  Hurwitz: 


det 


M(y)-s/ 


F 


0 


0,  y  e  r 


PROOF:  The  reciprocal  form  of  (99)  is 


(99) 
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0,(5)  det[/T.5i  - /<(i-)1“'B(»)]  “0,  r  e  r 


(100) 


where  a,(j),  defined  by  (73),  is  the  deienninsnt  of  [s/-A(y)]  which  is  the  open-looi>  characteristic  polynomial  of 
i4(y).  Since  Assumption  I  holds  there  is  a  coordinate  transformation  T^y) 

z  -  T(y)x  (101) 

which  takes  (1)  into  a  controllable  companion  form,  |34), 

i  -  A^a(y))z  +  BjUfx)  (102) 

where 

>f5(o(y))  -  7-(y)^(y)7-*(y)  (103fl) 

bay)  -  7-(y)B(y)  (103b) 


The  mam  matru  B,(r)  is  formed  from  m  columns  of  the  turn  identity  matrix.  The  dependence  of  B,(y)  on  the  uncertainty 
y  follows  from  the  fact  that  the  selection  of  the  m  columns  may  depend  on  yeP.  The  nxn  matrix  A,(a(y))  is  in  block 
controllable  companion  form.  Such  companion  forms  are  described  in  |32)*|34|.  In  view  of  the  Transformation  (101) 
we  can  rewrite  ( 100)  as 


det[[rr’(y)]  <i,(r)  r(y)Cr/-^(y)r'5(y)]  -0 


Consider  the  last  two  factors 


which  in  z-coordinates  is  given  by 


which  is  equivalem  to 


o,(s)  [nyKst-A(y)]-’B(y)] 


a,(y)  [[yf  -  ^,(«(y))]~'*j(y)] 


Ad/[s/  -  A/a(y))]B/y) 


(104) 

(105) 

(106) 

(107) 


where  Adj  is  the  matrix  adjoint  operation.  Consider  the  definition  of  an  nxn  symmetric,  positive  definite  matrix  P(y)  and 
the  definition  of  an  mxm  symmetric,  positive  definite  matrix  B(y)  such  that 

B-'(y)Bf(y)P(y)T(y)  -  f ,  y  .  T  (108, 

That  is,  Bfy)  must  be  such  that 

B-'{y)Bf{y)P{y)  -  f  r-'(y).  y  e  T  (109) 


Furthennore,  consider  the  Lyapunov  equation 


+  Afiiirmy)  +  e(y)  -  0.  y  *  r  (110) 

where  Q(y)  >  0  and  ^y) »  Q’'(y}.  y  c  P.  A  necessary  and  sufficient  condition  that  A,(a(y))  be  asymptotically  stable  and 
P(y)  be  symmetric,  positive  definite  and  satisfy  the  constraint  (109)  is  that  the  determmant  of  the  following  rruim  matrix 
( 1 1  la)  be  Hurwitz  and  that  the  following  mxm  matrix  (1 1  lb)  be  positive  definite  ,  132): 

Sj'(y)P(y)A4tjf-Aj^y)]BM  y  ,  T  (111a) 

Bj'(y)P(ymy)  >  0.  y  <  T  (1116) 

From  ( 104),  (107),  ( 109)  and  ( 1 1 1)  it  follows  that  (99)  is  necessary  and  sufficient  in  order  that  for  each  yeP  there  exist  a 
syrrunettic,  positive  definite  f(y)  satisfying  (109)  and  a  stable  /4,(^y))  such  that  the  Lyapunov  equation  (1 10)  is  satisfied. 
The  theorem  now  follows  from  Theorem  1.  Define  a(y),  y  e  P 


-  i8/'(y)[A^a(y))--4^a(y))]  (112) 

By  the  canonical  form  of  A,  and  B,  it  follows  that 

'4j(a(v))  *  A/a(y))  -  Sf(r)(T(y)  (113) 

Define  X(y),  y  e  P,  as 


^(y)-<r(y)r(y)  (114) 

Transforming  (113)  from  z*coordinates  to  x-coordinates  using  (101)  yields  the  following  asymptotically  stable  matrix. 

i<(y)  -  ^(r)  -  ^(rWy)  (115) 

Thus  condition  (i)  of  Assumption  II  is  met.  Transforming  (108)  from  z-coordinates  to  x-coordinates  using  (101)  gives 


F  -  R-'(y)B^(y)P(y) 

(116) 

where  F(y)  satisfies  the  Lyapunov  equation  which  is  transformed  from  (1 10) 

F(y)A (y)  +  A^(y)F(y)  +  C(y) - 0,  y  s  P 

(117) 

where 

F(y)-r^y)F(y)T(y) 

(118a) 

Q{y)~T^{ym)ny) 

(IIW) 
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Thus  condition  (ii)  of  Assumption  II  is  met.  Consequently  all  conditions  of  Theoitm  1  are  satisfied.  The  existence  of 
a  robust  sliding  mode  controller  (7)  now  follows. 

The  existence  of  a  robust  stabilizing  sliding  mode  controller  implies  the  existence  of  a  robust  linear  controlleT.  This  result 
is  given  in  the  next  theorem  which  parallels  the  scalar  result.  Theorem  7: 

THEOREM  9;  The  existence  of  a  subilizing  sliding  mode  controller  (7)  for  system  ( 1)  implies  the  existence  of  a  robust 
stabilizing  linear  controller 


u  -  -Kx  (119) 

PROOF:  A  necessary  and  sufficient  condition  for  the  evistence  of  a  robust  stabilizing  linear  controller  is  that  the  de¬ 
terminant 


-AM 


-K 


y  e  r 


(120) 


is  Hurwitz  where  /,  is  the  mxm  identity  matrix.  Paralleling  the  developement  (81)  •  (84)  the  determinant  (120)  is  Hurwitz 
if.  and  only  if.  the  mxm  matrix 

<<(r)  -  a,(j)  detC/„  -t-  K  Zsl  -  /ffy)]"'  R(y)].  y  e  T  (121) 


is  Hurwitz.  If  a  robust  stabilizing  sliding  mode  controller  (7)  exists  then  there  exists  an  nuui  matrix  F  such  that  (99)  is 
Hurwitz.  Consequently,  (100)  is  Hurwitz.  For  an  arbitrary  mxm  matrix  C  define  the  gain  matrix 


K  ~  CF 


(122) 


substitution  of  (122)  into  (121)  gives 

(i,'(j)  -  <i,(j)  detC/„  -I-  CF  [j/  -  A(y)]-'  %)].  y  t  r  (123) 


In  view  of  (73)  we  can  rewrite  ( 123)  as 
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-  <klCr.„  +  CFAdiU  -  ^(,)]  %)  +  (<I,W  -  >!")/„3.  y  e  T  (124) 

Since  the  inxm  nutiix  (100)  is  Hunvitz,  it  follows  that  there  exists  an  mxnt  matrix  C  with  sufficiently  large  elements' 
such  that  (124)  is  Hurwitz,|33|.  The  last  term  is  dominated  by  the  second  term.  The  control  law  (119)  robustly  stabilizes 
(1)  for  a  *  sufficiently  large' C  matrix  in  (122). 

7.  SUMMARY 

A  linear  time-invariant  uncertain  system  is  investigated  for  robust  stabilization.  The  uncertainties  belong  to  a  compact 
subset  of  multi-dimensional  Euclidean  space.  The  dynamics  and  iiq>ut  matrices  are  continuous  functions  of  uncertainty. 
The  system  is  controllable  for  each  uncertainty.  Assumption  1.  In  Assumption  II  two  general  conditions  are  stated  which 
involve  an  uncertain  Lyapunov  equation.  The  first  condition  deals  with  the  existence  of  an  uncenam  gam  matrix  for 
stabilizing  the  system.  The  second  deals  with  the  existence  of  a  constant  F  matrix  which  has  the  appearance  of  a  Riccati 
gain  matrix.  F  is  the  product  ot  three  uncertain  quantities  one  of  which  is  the  uncertain  sohilion  F(y)  of  the  Lyapunov 
equation.  Another  is  the  R{y)  matrix  which  is  assumed  in  Assumption  ill  to  form  a  positive  defirtite  matrix  when  added 
to  its  transpose. 

A  general  robustness  result  is  esublished  in  Theorem  1.  It  states  that  a  robust  stabilizing  sliding  mode  controL.  exists 
under  the  general  Assumptions  I  -  III.  In  Theorem  2  we  prove  that  the  matching  conditions  of  uncertain  systems  satisfy 
the  Assumptions  I  -  III. 

Robust  control  in  the  absence  of  matching  conditions  is  examined  in  Theorems  3,  4  and  S  for  scalar  control  input.  For 
such  systems  necessary  and  sufficient  conditions  are  given  for  the  existence  of  robust  stabilizing  sliding  mode  controllers. 
In  Theorem  4  we  show  that  systems  satisfying  such  conditions  also  meet  Assumptioru  I  -  III.  Theorem  5  goes  one  step 
further  and  shows  the  existence  of  a  robust  linear  control  for  such  systems.  The  existence  of  a  robust  sliding  mode 
controller  is  shown  to  depend  on  a  minimum  phase  condition,  Theorem  6.  In  Section  i  we  give  an  example  of  a  simple 
system  which  admits  a  robust  linear  controller  but  no  robust  sliding  mode  controller  that  stabilizes  the  system. 

In  Section  6  we  investigate  robust  control  in  the  absence  of  matching  conditions  for  multi-input  systems.  In  Theorem 
8  we  show  that  a  certain  determinant  being  Hurwitz  is  necessary  and  sufficient  for  the  existence  of  a  sliding  mode  con¬ 
troller.  A  sifflilar  condition  is  stated  in  Theorem  9  for  the  existertce  of  a  robust  linear  controller. 
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ABSTRACT 

A  dynamic  output  feedback  strategy  is  proposed  for  a  class  of  uncertain  systems.  Using  a 
singular  peiturbation  approach,  a  threshold  measure  “fastness”  (rf  the  feedback  dynamics,  to  ensure 
overall  system  stability,  is  derived.  This  tbresht^  is  calculable  in  terms  of  known  booitds  on  the 
system  uncertainties  but  may  be  cooaervative  in  piactice.  To  circumvent  this  drawback  and  to  allow 
for  bounded  uncertainties  with  unknown  bounds,  an  adaptive  version  of  the  strategy  is  then 
developed. 


1.  Introduction 

We  address  the  problem  of  design  of  dynamic  output  feedback  controls  for  a  class  of  unceitain  nonlinearly 
pertuibed  linear  multivatihble  systems.  The  af^roach  is  similar  in  concept  to  that  of  [1],  and  fundamentally 
stems  from  ttie  deterministic  theoty  developed  in,  for  examine,  [2-8]  (see  also  bibliographies  therein). 

Initially  considering  a  hypothetical  output  y*  for  the  system,  a  (generally  unrealizable)  stabilizing  static  output 
feedback  control  is  established.  This  static  control  is  then  approximated  by  a  realizable  compensator  (with 
parameter  /r  2  0)  which  filters  the  true  system  output  y.  Physically,  the  parameter  is  a  measure  of  "fastness" 
for  the  filter  dynamics;  analytically,  m  plays  the  role  of  a  singular  perturbation  parameter.  Using  a  singular  per¬ 
turbation  analysis  akin  to  that  of  [9,10],  a  threshold  measure  ft*  of  "fastness”  of  the  compensator  dyiuunics,  to 
ensure  overall  system  stability,  is  then  derived.  The  threshold  is  explicitly  calculable  from  known  system  data 
but  corresponds  to  a  "worst-case"  value  and  consequently  may  be  conservative.  To  counteract  this  inheiem 
conservatism  (and  to  allow  for  bounded  uncertainties  with  unknown  bounds)  an  adaptive  version  of  the  com¬ 
pensator  is  also  developed  by  an  approach  which  is  essentially  that  of  [11]  (see  also  [12-16]  and  related  work 
in  [17-23]). 


2.  The  system 

We  consider  unceitain  nonlinearly  perturbed  linear  systems  of  the  form 

i(f)  *  Ax(r)  +  B[«(r)  +  g(f,z(r).ii(r))).  z(0eJf*.  ii(f)  e  If"  (1) 

for  which  the  only  available  state  information  is  provided  by  the  output 

y(r)  »  Cx(f),  y(»)  c  K',  mipi  n  . 

The  triple  which  defines  the  ixirainal  linear  system,  Is  sssumed  to  satisiy  the  foUowiitg. 


(2) 
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A 


Assumption  I:  (,A,B)  is  a  coDOoUable  pair  and  rank  S  ^  m. 

Assumption  2: 

TTiere  exist  known  integer  r  2  1  and  known  matrices  Fi  .Fj,-  •  -.F,  e  such  that 

(i)  for  I  *  1,2,- •  r-l.  im  Cy4‘"'5  c  n  kerf;  ; 

^i+1 

moreover,  the  matrix  C,  :=  FiC  -t-  FiCA  +  •  •  •  +  F,CA'~^  is  such  that 

(ii)  |C,B|  +  O.and 

(iii)  the  transmission  zeros  of  the  m-input  m-output  linear  system  (Cr,A,B)  lie  in  C~  (the  open  left  half  com¬ 
plex  plane). 


Example  );  If  A  = 

‘o  1  o' 
0  0  1 

.  B  = 

o' 

0 

.  C  = 

10  0 

0  0  1 

0  0  0 

1 

r  =  2,  f,  =  (1  1]  and  fj  =  [1  0]. 


,  then  the  above  assumptions 


hold  with 


Finally,  we  impose  some  smicnire  on  the  uncertain  function  g. 


Assumption  3: 

g:  KxSCxIB'"  -»  R"  is  (i)  Carathdodoty,  with  (ii)  Hg(r,x,u)||  S  flt||x||  +  ^||«||  for  all  (r,x,«),  where  a 
and  are  known  constants  with  p  <  1,  and  (iii)  if  r  2  2.  then  g  is  uniformly  Lipschitz  in  its  final  argument 
(with  known  UpschiB  constant  A),  i.e.  (if  r  2  2)  diere  exists  known  i,  independent  of  (t,x),  such  that,  for  all 
u  and  V,  ||g(r,x,«)-g(f,x,v)||  S  ,l|i«-v||. 


The  outline  of  the  paper  is  as  follows; 

Fustly,  the  problem  of  designing  a  (dynamic)  output  feedback  compensator  for  system  (1,2)  is  addressed.  This 
is  accomplished  by  initially  considering  system  (1)  with  hypothetical  output 

y»(f)  =  CrX(i)  (3) 

where  Cr  is  as  in  Assumption  2.  Note  that,  if  r  s  I  then  y*(r)  «  F|y(r)  and  hence  is  realizable;  however,  if 
r  2  2  then  y*(t)  is  unavailable  to  the  controller,  hence  the  qualifier  "hypothetical".  For  the  system  (1.3)  so 
defined,  (ii)  and  (iii)  of  Assumption  2  in  essence  play  the  role  of  "relative  degree  one”  and  "minunum  phase" 
condinons  on  the  hypothetical  nominal  linear  system  triple  (Cr.A.B).  Under  such  conditions,  it  is  kix>wn  (see. 
for  example,  [11-13])  that  the  zero  state  of  system  (13)  can  be  rendered  globally  unifoimly  asymptotically 
stable  by  static  output  feedback;  this  is  reiterated  in  Theorem  1.  However,  with  the  exception  of  the  case 
r  *  1,  such  sutic  output  feedback  is  unrealizable  in  the  context  of  the  tree  system  (13).  Therefbre.  in  §3.  a 
realizable  dynamic  compensator  is  constructed  for  the  cases  r  2  2,  which  filters  the  actual  output  y.  This  filter 
can  be  interpreted  u  providing  a  realizable  approximatioa  lo  the  static  hypothetical  output  feedback;  moreover, 
it  is  shown  in  Thetnem  2  that  global  uniform  asymptotic  stability  of  the  zero  state  of  (13)  is  guaranteed  pro¬ 
vided  that  the  filter  dynamics  are  sufficiently  fast  (a  calculable  threshold  measure  of  fastness  is  provided). 
Secondly,  in  }4,  an  adaptive  version  of  the  dynamic  compensator  it  developed,  which  counteracts  conservatism 
(induced  by  crude  estimates  in  the  analysis)  inhereta  in  the  non-tdaptive  filter  and  which  also  dispenses  with 
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the  requirement  that  tte  tnoeniiiity  paianieten  a,  fad  X\n  Aannipdon  3  be  known  (however,  the  assump¬ 
tion  that  fi<  \  remains  in  (broe  and,  moreover,  if  r  h  2  then  g  is  assumed  to  depend  Uneaiiy  on  x). 


3.  Stabilizing  static  output  feedback  control  Itor  h;potlictkal  system 
Let  T|  €  be  such  that  ker  Tj  a  im  fi.  then 

with  inveise  7“*  »  (S,  ;  S] 


is  a  similarity  transfoimation  which  takes  system  (13)  into  the  form 

i(t)  a  A„S(0  +  412^(0  .  jr(t)  €  «•-"  (4a) 

^•(r)  a  42,jr(/)  +  4225^(0  +  u(0  +  f(t.X(t).y(t).u(t))  .  yU)  e  K”  (4b) 

l{t,X.y.u)  g(t,SiX-t-By,u)  (4c) 

with  output 

y*(t)  a  (,C^)srU)  .  (5) 


Note  that  the  eigenvalues  of  4ii  coincide  with  the  transmission  aros  of  (C,,A,B);  thus,  by  virtue  of  Assump¬ 
tion  2(iii),  ff(4n)  c  C~. 

Let  /*!  >  0  be  the  unique  positive  definite  solution  of  the  Lyapunov  equation 

PiAu  +  4,V,  +  /  a  0  (6) 

then  we  state  our  first  result 

Theorem  I: 

Define  k’  :*  II422II  +  ol|fll|  +  J  Ill/’i4i2+4fi  J+a85i  II]*  .  then,  for  each  fixed  6  >  «r’(l-/J)"',  the  static 
output  feedback 

a(»)  a  (C^)-'y*(t)  -  -If  m  (7) 

renden  the  zero  state  of  the  hypothetical  system  (1,3)  gkfoaily  unifonnly  asymptotically  staUe. 

Proof:  Let  V;  {X.f) )-»  { {X,P^X)  +  then  a  straightft»waid  calculation  reveals  that,  along  solutions 
(X(')>y('))  of  (43,7)  (equivalent  to  (1,3,7)),  foe  following  holds  almost  everywhere 

j  v(m.m)  s  -mx(t).y(t)) 

where 


/ 

[jr' 

X 

1  -{yi‘i4„+4£|+a|S,|] 

\ 

[9. 

1  M 

.9. 

/  .  Af  :■ 

-(|/'i4„+42’i»+o|S,  |]  2I#(l-^)-|42,|-a|fl|) 

Noting  the  Af  is  positive  definite,  the  result  follows.  □ 


7-1 

.(.CrBT'Cr 


/ 
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bi  tte  Motext  of  the  tnie  sy$ia&  (U).  if  r  -  1.  then  die  ititk  ftecibtdc  (7)  b  lea^^ 

«(,) «  -^eiCrBT'Pxyit)  (8) 

wbence:- 


CoroUary  1: 

Let  if  be  as  in  Theorem  1.  If  r  «  I  then  the  static  out{«t  ftedback  (S)  renden  the  zero  state  of  the  tiue  system 
(1^)  globally  unifonnly  asymptotically  stable. 


However,  in  all  other  cases  (r  2  2),  the  feedback  (7)  is  unrealizatde  for  the  itue  system  (1,2);  in  its  place,  we 
win  develop  a  realizable  dynamic  compensator  in  the  next  section. 


4.  Cases  r  2  2:  Stabilizing  dynamic  output  feedback  for  the  true  system  (1,2) 

In  view  of  Assumption  2(i),  we  note  that 

y*(t)  =  C,x(t)  =  F,y(0  +  F2y(0  +  •  ■  •  +  Fry^"Ht) 
which  can  be  inteipreted  in  the  fiequency  domain  as 

y*(s)  =  [Fi+W(s)ly(i)  . 


where 

N(s)  ■  SF2  +  •  •  •  +  s'~^F, 

is  physically  unrealizable.  Our  approach  is  to  replace  N(s)  by  a  physicaUy  realizable  transfer  matrix  (filter)  of 
the  fonn  H^(s)N(s)  with  appropriately  chosen  H^U)-  To  this  end.  let  d,-  S  r-1  denote  the  (tegree  of  the 
highest-degree  polynomial  in  the  ith  row  of  Af(j).  Let  constants  aj  >  0,  ;=2,-  ■  •,</,•,  be  such  that 

*,(j)  »  s'*'  +  +  •  •  •  +  OjS  +  1,  i  ■  1,2,-  •  -.m 


is  Hurwitz  (i.e.  with  all  its  roots  lying  in  the  open  left  half  complex  plane  C").  For 
h^(s),  parameterized  by  ^  >  0,  as 


hf(f) 


1 

*((/«) 


1,2,  ■  ■  ■  ,m  ,  define 


which,  inteipreted  as  a  transfer  function,  has  minimal  realization  (c,^ ,  ti~'h,),  where 


P 

0 

1 

0  •  ■  • 

• 

0 

o' 

•  • 

1 

Ai  - 

0 

0 

1  • 

0 

h.  - 

0 

c  ,  e,  - 

0 

b 

0 

b  •  •  • 

1 

b 

b 

-1 

-ai 

-ai  ••• 

-ai 

J 

1 

.0 

We  now  introduce  the  transfer  matrix 


H^(s)  :»  dUg(Af(f)) 


wUdi  deady  bas  minbiial  fealizaion  (C* . /t'U* .  where 

it*  -  diag  M,)  <  .  B*  -  diag  (b,)  «  .  C*  -  (Bag  id]  m  *"*«  .  wiih^  £di  . 

We  note,  in  passing,  that  o(.A*)  c  C~  and  that  C*(d*)~*B*  ■  -/. 

Let  ar*  be  as  in  Theoiein  1,  then,  for  fixed  6  >  the  proposed  ptaydcaUy  realizable  compensator 

(which  filters  the  actual  output  y)  for  system  (1.2)  is  parameteriied  by  ti,  and  has  ftequency  domain  diaiacteri- 
zation: 

G^(s)  -  -tiC,BT^lFi+H,(.s)N(.s)]  .  (9) 

Fbr  ootational  convenience  we  introduce  functions  p.fi,f2.  A/}  and/j,  defined  as  follows. 

p:  (JT.jr.f)  H*  -ie(C,B)-‘  +  C*?] 

/,:  (ir.jO  +  AijjT 

/i:  (t.jf.jO  (-»  Ai,Jf  +  Aliy  -  fy  +  #(r.jf.y.-tfy) 

A/j:  (/.i-.y.?)  t-»  fy  +  «»(f.y.2)  +  g{t,x.9,p(x.9.n)  -  ?(t.«.y.-^y) 

/s :  (i.y.y)  t-t  A*?  +  B*  [  c,fly-f',C(S,g+fly]3 . 

Then  it  is  readily  verified  that,  in  die  time  domain  and  under  state  transfoimation  T,  the  differential  equations 
governing  the  dynamic  ouqxit  feedback  controiled  system  may  now  be  expressed  in  the  fotm: 

i(f)  =  /t(XUhnO)  .  XU)  e  Jf**"  (10a) 

9U)  -  /j(f.*(f).y(t))  +  AfiU.xu)jU).iU)) .  y(f)  e  dot) 

niU)  -  fzWO.m.XU))  .  ?(»)  e  .  (10c) 

In  analysing  the  stability  of  this  system,  we  regard  /i  as  a  singular  peituibation  parameter.  Recalling  that 
C’(A')~'B*  a  -/,  we  note  that  system  (4)  with  control  (7)  is  recovered  on  setting  /i^Oia  (10);  thus,  in  the 
usual  tenninology  [9,10,24],  system  (4,7)  may  be  inteipreted  as  the  reduced-order  system  associated  with  the 
singularly  perturbed  system  (10).  The  ensuing  approach  is  akin  to  that  of  [9,10],  our  objective  being  to  deter- 
miiK  a  threshold  value  ft'  >  0  such  that,  fbr  all  p  c  (P./t'X  Hk  zero  state  of  system  (10)  is  globally  uni- 
foimly  asymptotically  stable. 

Recalling  that  <r(A*)  c  C~,  let  B*  >  0  be  die  unique  symmetric  positive  definite  solution  of  the  Lytqwnov 


equation 

P'A’  +  (A' fP'  *lmO. 

(11) 

Define  W:  -♦  [0,»»)  by 

W(t.y.y) i<w(x.y.j),  B*w(t.y,f)> 

(12a) 

where 

w(t.y.D  y+  (A*)-'B*[c,By-B,ci5,jr-»-By]] 

-  (A*)-73(jr.y.f) . 

(12b) 

We  now  establirii  some  preUminafy  lemmas.- 
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Hk  &st  is  implicit  in  tiK  pnxrf  of  Theorem  1. 

Lemma  1: 

<V,V(t,y)y,ajO>  +  <V,V'(f.y)./j(/jr.y)>  i  -a^V{,S,f)  where  [|lAr‘|[|i’il+ll]-‘  >  0. 

Lanma  2:  <V,V(j:.y.5)./3(X.y.J)>  S  -AW<T.y.f)  «*ere  A)  li*!'*  >  0  . 

Proqf:  <v,iy(j'.y.f)./,(i'.y.j)>-</»*wtf.y.*)./3(jr.yi*y)> 

-  (P'w(x,y,r).  i4V(jr.y,f)> 

-  -  i 

Qeariy,  the  function  I/i  I  is  bounded  above  by  a  calculable  scalar  multii^  of  the  fimction  Vl.  In  view  of 
Assumption  30i),  ll/j  H  is  also  bounded  above  by  a  calculable  scalar  multi|de  of  V^.  By  Assumption  3(iii),  g  is 
unifonnly  Lipscfaia  in  its  final  aisument  (with  known  Lipschiiz  constant  l)i  beau, 

IIA/2(t.y.y.?)ll  S  (i+>i)|/e  y  +  p(y.y.f)il  for  au  (t,x.y.r> 

and,  since  £  y  +  f(y,y,f)  «  -£(C,B)~^C‘w(X,g.Z),  it  follows  that  IA/2I  is  bounded  above  by  a  calculatde 
scalar  mult^  of  Therefore,  we  may  conclude: 

Lemma  3: 

There  exist  calculable  constants  0o,  ar,,  and  no  that,  for  all  (t,X,y.D, 

0)  <v,vi^(x,y,?),  /,(x,y)>  i  ^vi(x,y)ivi(x,y,r) . 

Cii)  <v,jy(x,y,2),  /j(r,x,y)  +  A/2(r,x.y,D>  s  wi . 

Ciii)  <  S  >toV^(.x,sr)wHx,y,r) . 

The  next  theorem  demonstrates  that  system  (10)  is  asymptotically  stable  for  all  n  >  0  sufficiently  small 
Theorem  2: 

Let  r*  be  as  in  Theorem  1  and  define  a"  >  0-  Then,  for  each  fixed 

6  >  and  fixed  a  e  (0,/<*),  the  zero  state  of  system  (10)  is  globally  unifonnly  asymptotically 

stable. 


Proof:  Define  the  positive  definite  quadratic  fonn  (Lyapunov  fimction  candidate)  "W  by 

-MJf.y.f) m,y)  +  [ib+rjr'oomy.n 


then,  invoking  Lemmas  1, 2  and  3,  the  following  holds  almost  eveiywhere  ahmg  solutions  (X(’)>y(').f('))  of 
(10): 


Vi(X(t).m) 

,"'*(jr(f).y(0.r(/)). 


y*(t(t).f(0)  ]\ 
.»»'*(«o.y(t).it»))  J' 


/ 
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wbeie 


4g  -no 

.-no  (/«~‘A)-riX^+W2r‘>7o. 


Nbdng  ibat  *(  is  positive  definite,  the  result  follows.  □ 


In  summaiy,  let  >  (J,  realize  (minimally)  the  component  HJ,s)N{s)  of  the  proposed  com¬ 

pensator  (9),  then  the  overall  controlled  system  has  the  stnicture  shown  in  Hgure  1. 


compensator 


Figure  1 


The  goyeming  equations  (equivalent  to  (10))  can  be  expressed  as 

i(f)  -  Ax(t)  +  BluO)  +  g(t.x(/).«(0)l.  x(f)  «  K*  (13a) 

/«(»)»  Jte(«)  +  ®y(0.  z(0«R*.  p<m'.  (13b) 

y(0  -  Cx(t)  €  St"  (13c) 

«(0  »  -/e(C,a)-'tf,y(0  +  ^z(O)  «  K"  .  «  >  x*(l-^)''  .  (13d) 


aeariy,  the  threshold  values  r*  and  /<*  are  central  to  this  desiga  Since  these  values  are  deteimined  via  a 
"worst-case"  analysis,  it  is  to  be  expected  that,  in  ptactice,  the  compensator  will  be  conservative.  In  the  next 
section,  a  stabilizing  adaptive  version  of  the  compensator  is  developed;  however,  in  dre  case  r  2  2,  this  is 
adiieved  at  the  expense  of  imposing  ftuther  stnicture  on  the  uncenain  function  g. 


S.  Adaptive  compensator 
5.1  Case  I:  r*l 

If  Assumption  2  holds  with  r  «  1  then,  by  Cofollaiy  I.  system  (1,2)  is  asymptodcaOy  ttabiliwid  by  the  static 
output  feedback  (8)  with  6  >  /r*(l-^)~'  provided,  of  course,  that  Fi  nd  C,B  are  known  and  that  sufficient  a 
priori  inibnnaiion  is  svilaUe  to  compute  the  (conservative)  gain  threshold  We  now  consider  the 

case  Ibr  which  the  latter  infisimatian  is  unavailatde,  i.e.  we  only  assume  knowledge  of  fi  aid  C^B  and,  in 
pankular,  the  constants  a  snd  id  <  1  in  Assumption  3  may  be  unknown.  AH  other  assumpdons  remain  in 
(brea. 
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Rqjlace  fixed  in  (8)  by  variable  ir(i)  to  yield 

u(t)  m  ^KOXCrBT'Fiyit)  (14a) 

and  let  ir(0  evolve  according  to  the  adaptation  law 

#(»)  -  ||(C^)-‘/f,y(t)|*  .  (14b) 

then:- 

Theorem  3: 

For  all  initial  data  (<b.x((t))>x(i6))  e  8?xJt*x[0,~).  the  adaptively  controlled  system  (1^,14)  exhibits  the  fol¬ 
lowing  propeities: 

(i)  lim  x(t)  exists  and  is  finite: 

(ii)  lim  llx(OII  =  0. 


Proof:  FOr  fixed  (but  unknown)  6  >  ir*(l-fi)~'  and  under  the  similarity  tiansfoimation  T,  system  (1^,14) 
may  be  expressed  as 

iU)  =  -4„i'(r)  +  A^jSit)  (15a) 

^(f)  =  42,jr(f)  +  ArfSU)  -  69U)  -  («r(f)-/e)y(f)  +  ?(t.«»).y(0.-«r(t)y(t))  dSb) 

Ht)  »  liy(t)ll*  .  (150 

Letf/andVbeasintbe  proof  of  Theorem  1  and  define  the  positive  definite  (since  ^  <  1)  function 
V:  (JfJ.jr)  I-*  V{X,f)  +  ^K~li9-iP{K-e)\K-li\  . 

Then,  along  solutions  (i'(*).y(‘)<«'('»  of  (15),  the  following  holds  almost  everywhere 

s  -u(x(t).nt))  -  pemoi^  -  dcw-pymof  +  fix(r)mr)ii* 

of 

+  ((r(t)-/e)-)»|  «f(f)-f  I  ]|iy(t)»*  • 

s  -i;(*(t).y(i)) ,  (16) 

Since  1/  is  positive  definite,  we  conclude  that  r  i-»  (Jf(r),^(r),x(r))  is  bounded  and  since  1 1-»  e(t)  is  also 
monotonic,  assertion  Q)  of  the  theorem  (OUows.  Rntliennore,  in  view  of  (16),  we  have 
j^U(X(t).S'(t))dt  S  tXXihJWMh))  <  •*  and  hence,  since  (/  and  V  ate  positive  definite  tpiadratic  fixms, 
j^V(jr(r),7(r))dr  <  moreover,  V(X(.‘),y{’))  is  essentially  bounded  from  above.  Tbetefine,  we  conclude 
that  V'(X((),y(t))  -*  Out  —  (see  »*«"«"«  6.3  of  [22]),  whence  assertion  Qi)  of  the  theoiem.  □ 


52  Case  tt:  ri2 

Beibre  describing  the  adaptive  strategy  in  tiiis  case,  it  is  remufeed  that  the  argument  used  in  estahiititing 
Theorem  3  cannot  be  carried  over  directly.  Instead,  we  will  base  our  approach  on  that  of  Minnsaon  [11]. 
FOr  tUs  reason.  Anther  conditions  are  imposed  on  the  uncertain  fraction  g.  In  particular.  Assumption  3  is  now 
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replaced  by: 

Assumption  3'; 

There  exist  a  bounded  continuous  ftmcdon  K  -*  a  Canthdodoty  fiinction  :  Kx-K”  J?" 

which  is  unifoimty  Upschitz  in  its  second  aisument,  and  a  constant  fi  <  I  such  that 
0)  -  AA(t)x  +  &(i,b).  flw  all  {ta.u). 

Cii)  i  ^l|«ll.  fiw  an  (».«). 

and 

Oii)  (Cji+BAA(-))  is  unifomily  comidetBly  observable  in  the  sense  of  [2S]. 

Note  that,  if  Assumption  3'  holds,  then  Assumption  3  holds  a  fortiori  with  a  >  sup  |dA(t)0  provided  that  a, 

and  the  Upschitz  constam  for  ;«(<,■)  are  known.  However,  knoudedge  of  these  constants  is  not  required 
here. 


Example  2:  With  (C.A.B)  defined  as  in  Example  1  of  §2.  Assumption  3'(i)  holds  for  any  bounded  continuous 
AA:  t  i-»  (Aai(t),A<ij(O.A<i3(t)). 


Now  replace  fixed  if  in  (13d)  by  variable  ir(t)  >  0  and  ie{dace  fixed  p  in  (13b)  by  (&r(t))"’,  where  >  0  is 
a  constant  (design  parameter)  and  let  tc(t)  evolve  acconSBg  to  the  adiqxaiion  law  (other  adaptation  laws  may 
be  feasible,  as  discussed  in  [20]) 

at)  =  8y(t)ll*  +  l*(08*  . 

Writing  (as  in  [11]) 


then  the  overall  adaptively  controlled  system  may  be  expressed  in  the  fonn 


xHt)  -  A^(/)x»(/)  +  fi^[«^/)  +  g^(/,i»^r))]  .  x\t)  e  , 
yHt)  -  C^xHt)  €  , 

u\t)  -  -KU)K^yHt)  €  , 

-f(0  -  . 

where 


(Ha) 

(17b) 

(17c) 

(17d) 

(17e) 

(17f) 


/ 
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1116  Stability  of  system  (17)  will  now  be  investigated.  We  first  require  tbe  following  fenima  (essenfially  a  non- 
autoncmous  veision  of  MSnensson’s  lemma  (HD- 

Lemma  4: 

Let  ;  Ji  -»  J{***  satisfy 

x\t)  m  +  Bhm  + 

where  v:  tt  -*  1?"**  is  measurable.  Then,  for  each  fixed  r  >  0  there  exists  a  constant  e  >  0  such  that,  for 
all  t, 

l|xt(/)|l*  S  c  /^[llyns)ll^  +  lv(011^]  ds  . 


Pro<rf:  Let  ^■,*)  denote  the  stale  transition  matrix  fimction  generated  hy  A+BAA(')  and  define  the  observabil¬ 
ity  Gramian  for  tbe  pair  (C.A+BAAO))  in  the  usual  manner,  that  is, 

r(f,s)  :=  jV(<r,i)C’‘C®(<r,i)  da  . 

Now,  for  some  constants  ki  and  oi,  we  have  ||exp  i4t||  i  kit"  and,  since  M(‘)  is  bounded  (by  assumption), 
there  exists  constant  ki  such  that  ||SM(()||  2  ki.  By  standard  peimibation  theory,  we  conclude  that 

ll<I>(r,j)||  £,  for  all  t.s  . 

Dearly,  the  state  transition  matrix  function  <b^(*,*)  generated  by  yl^(‘)  is  given  by 


®(t,s)  0 
0  I 


whence 


where 


||^’(f,r)|l  S  Ci(r-j)  for  aU  t.s. 


c, :  <r  i-»  1  +  *,e<'**‘*^' 

The  observability  Gramian  for  the  pair  (C^,i4^('))  is  given  by 


(18a) 


(I8b) 


r^(r,s)  ;* 


r(».j)  0 

0  (t-s)/ 


and,  since  (C,A+BAA(.‘))  is  unifonnly  completely  observable  (by  assumption),  we  may  conclude  (see  [25]) 
that,  for  each  fixed  r  2  0,  there  exist  positive  omstants  C2  and  Cj  such  that,  for  all  t. 

Cl  Ilf  f  S  <f .  r^(r.r-T)f>  S  c,  Ifll*  V  f  €  J?-»  .  (19) 

Now  define  the  measuralde  function  ;  1 1->  v(t)+g^(r,y(i))  and  note  that  |v^(t)|  i  (l+^)l|v(t)|.  Then, 
x^(0  ■  0^(r.t-t)x^(t-r)  +  J^|^0^(f,j)fi^v^(j)  ds 

whence 

||xt«)|2  s  2|®^r.r-r)x^r-»)B*  +  2|J^<»»(r.s)ff%»(x)  dbl* 

i  2c4lxt(r-t)|*  +  2cj(l+/»»llS^|*J^|v(i)||»dr ,  (20a) 
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wherein  (18)  has  been  used,  and 

C4  cf(t).  Cs  :■  J^cf(j)  ds  .  (20b) 

Also,  invddng  both  (18)  and  (19), 

ljc^(f-f)ll^  i  cf'  (x^(»-T),r^(t,f-r)x^(/-T)> 

*  C2*J^I1  y^(J)-C^/^®*(j.a)i»N^<T)  dff  dll 

^  2c2‘'  [  J',!^ly^(i)ll*dJ  +  C4T(l+^)»|C^J^IlB^l|»/^|v(s)|*di]  .  (21a) 

where 

C(,  :*»  j^j^c}(a)d<Tds  .  (21b) 

Ck>mbining  (20)  and  (21)  yields  the  required  result  D 


Theorem  4: 

For  all  initial  data  (i(o,x^(to)-*'('b))  ®  system  (17)  exhibits  the  following  properties: 

(i)  lim  K(t)  exists  and  is  finite; 

l-4« 

(ii)  lim  |Ixt(t)J  *  0  . 


Proof:  Seeking  a  contradiction  to  (i),  suppose  that  die  mtmotonically  increasing  function  1 1-1  ir(r)  is 
unbounded.  Then,  tor  some  tj  «  (0,«*),  e(%+tj)  =  jP  >  x*(l— ^)"*  and  (diir(<b+ti))”*’  “  it  <  m'-  Now,  an 
argument  similar  to  that  used  in  the  proof  of  Theorem  2  can  be  adopted  to  establish  the  existence  of  a  positive 
definite  quadratic  form  x^  V^fx^)  and  positive  constant  p  such  that  the  following  holds  on  solutions 
(xt(0.x(*)):  (to.-)  -»  «*-^x(0,-)  of  (17): 

^  'I'^fx^fr))  s  -p  ■V'^(x^(r))  for  almost  aU  t  2  to  -t-ri  . 

Thus  x^ :  [to,—)  -»  Jl"**  ultimatdy  tends  exponentially  to  zero;  hence,  both  x^  and  arc  square  integrable 
on  [<6'**),  which,  in  view  of  (17d),  contradicts  our  supposition  diat  the  function  k  is  unbounded.  This  estab¬ 
lishes  assertion  (i)  of  the  theorem. 

It  remains  to  show  that  x^(/)  -»  0  as  r  CSeaily,  (i)  ensures  that  is  square  imegratde  cm  [«(),-)  and, 
in  view  of  (17c),  that  is  a  bounded  linear  transfonnation  of  y^ .  Thus,  we  may  conclude  that  is  also 
square  integrable.  Now,  by  Lemma  4,  we  have 

|x»(f)l*  s  c  J^,(ly^z)ll*  +  l«^(s)Il*l  ds 

-  c/^[|y^»)l*+l«nj)|^l  ds  -  cjl^ [ly^WI^+Bu^j)!*]  *  . 

Iherefbie,  ||x^(t)|  0  as  r  a 
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&  Discontiniioiu  feedback 

bi  tbU  find  seakxi.  sane  possible  genenliTadow  of  tbe  proposed  compensaiots  are  briefly  discussed.  In  [23] 
and  to  tbe  case  r  >  l  only,  a  wider  class  of  unceitabi  fencdais  g  is  studied;  ^edfically,  Assumption  3  Cu)  is 
iqiacedby  the  condition 

5  +  ^Hall  +  rf(C*)  to  all  (f,*,u) 

with  a  and  ^  <  1  as  before  and  where  r  is  a  constant  (assumed  known  in  the  non-adaptive  case)  and  { is  a 
known  continuous  fimctioa  Thus,  loosely  speaking,  in  [23]  a  non-cooe-bounded  componem  of  uncettainty  is 
allowed  but  this  is  required  to  be  bounded  by  a  functkm  of  the  system  output  y.  In  tbe  context  of  tins  more 
general  class  of  systems,  tbe  asseition  of  CoioUaiy  1  of  the  present  paper  remains  tree  for  fixed 
a  >  (l-/l)~‘max  (r* ,/)  if  (8)  is  replaced  by  the  geneialiaed  fieedbadc 

«(/)  6  -6  [(C^)-'F,y(t)  +  {(y(f))S«y(t))]  .  (22a) 


where  the  set-valued  map  y  i-»  ^y)  c  K”  in  essence  models  a  discontinuous  control  component  and  is 
given  by 


Xiy) 


(||(C^)-‘^,yr‘(C,fl)-'/-,y};  f,y  +  0 
(v:  ||v|Sl);  fiy»0. 


(22b) 


and  the  overall  controlled  system  is  consequently  imerpreted  in  tbe  generalized  sense  of  a  controlled  differen¬ 
tial  inclusion  [26].  Furtbeimore,  the  assertions  of  Theorem  3  of  the  presem  paper  remain  true  if  (22)  is 
replaced  by  tbe  adaptive  control 

u(t)  €  -K(t)[(CrSr^F,y  +  {(y(t))^(t))] 
where  /e(f)  evolves  according  to  (14b). 

In  tbe  cases  r  i  2,  preliminaiy  investigations  indicate  that  again  a  non-cone-bounded  component  of  uncettainty 
(although  considerably  less  general  than  that  of  tbe  preceding  paragraph)  can  be  tolerated  in  g  and  counteracted 
by  augmenting  tbe  compensator  (13d)  (or  its  adaptive  counterpart  implicit  in  (17c4))  with  an  appropriately 
chosen  set-valued  map  (^ain  essentially  modelling  a  discontinuous  control  componem).  However,  the 
requisite  stracturai  conditions  on  the  txm-cone-bounded  uncettainty  are,  as  might  be  expected,  of  a  rather  res¬ 
trictive  and  technical  nature  (aldn  to  those  in  [10])  and  are  not  detailed  here. 
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ABSTRACT 

Modelling  of  systems  Is  generally  done  by  frequency  response  methods  or  state 
variable  methods.  It  Is  our  object  to  show  how  frequency  domain  robustness 
results  can  be  extrapolated  to  their  state  space  counterpart.  Using  proper¬ 
ties  of  Input-output  relations  of  systems,  and  different  compatible  norms.  It 
will  be  shown  how  a  corresponding  frequency  response  robustness  results  can  be 
applied.  The  method  can  be  used  to  solve  a  certain  class  of  non  linear 
equations.  It  can  also  apply  to  the  control  of  non  linear  uncertain 
multivariable  systems  In  order  to  better  stability,  sensitivity  as  well  as 
decentralized  control  results.  It  can  also  apply  to  assess  the  state  feedback 
compensator,  the  observer  and  the  output  feedback  compensation  with  regard  to 
the  "obustness  problem. 

Multivariable  control  theory  evolved  in  the  sixties,  using  the  state  variable 
approach.  This  approach  together  with  growing  computer  technology  gave  rise 
to  tremendous  research.  Interesting  results  on  system  stability, 
controllability,  observability,  reachability  and  detectability  were  developed. 
This  was  a  sharp  contrast  to  the  single  input-single  output  frequency  response 
approach  involving  polynomial  approaches.  Nyqulst  criterlum,  and  root  locus 
methods. 

However,  many  of  the  answers  given  by  state  space  methods  lack  the  suppleness 
of  multivariable  methods  as  they  apply  to  well  defined  models  with  no 
modelling  uncertainty.  Adaptive  control  is  a  partial  response  for  the 
modelling  uncertainty  problem  as  far  as  parametric  uncertainty  Is  concerned. 
Clearly,  In  any  state  space  representation  (A.  B,  C,  D),  there  is  no  way  to 
predict  the  behaviour  of  eigenvalues  whenever  the  matrix  representation  is 
modified  to  (A-tAA,B,C,D).  On  the  other  hand,  frequency  response  methods  apply 
better  to  the  uncertainty  problem;  in  the  case  of  a  single  input  single  output 
Nyqulst  diagram  for  instance,  a  Nyqutst  plot  could  be  replaced  by  some  Nyqulst 
band  representing  the  modelling  uncertainty  at  each  frequency. 


Multivariable  frequency  reaponse  nethoda  auch  aa  the  inverae  Nyquiat  area  [1] 
multivariable  Nyquiat  crlterlum  [2]  and  multivariable  root  locua  [3]  are 
concerned  mainly  with  ayatem  atabllity.  However,  the  input  output  approach  to 
systems  1 4, S, 6, 7, 8]  which  apply  to  any  normed  algebraic  representation  of 
systems  fit  partlcularlly  to  the  frequency  response  setting.  Such  an  approach 
allows  us  to  handle  the  problem  of  m^elling  uncertainty.  It  is  our  purpose 
to  show  how  multivariable  frequency  response  uncertainty  methods  can  be 
extrapolated  to  the  multivariable  state  apace  uncertain  models  case. 

It  is  our  aim  to  show  how  these  input  output  robustness  results  can  be 
Implemented  in  aysteais  described  by  their  state  apace  form.  Given  a  state 
space  model  with  a  state  feedback  compensation,  observer  output  feedback 
compensation,  we  shall  derive  the  best  possible  bounds  on  the  closed  loop 
perturbations  due  to  some  uncertainty  4A  in  the  dynamics  of  a  system. 
Conversely,  any  frequency  response  robustness  result  will  be  shown  to  hold  for 
a  corresponding  state  space  disturbed  model  A  'f  4A  and  bounds  on  acceptable 
uncertainties  4A  will  be  deduced. 
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A  MODEL  PROBLEM.  We  consider  control  systems  with  deterministically  uncertain  dyna^ 

mics  described  by  the  differential  inclusions 

,  Sr  £Gj(t,x,x,y,?,Uj,U2) 

(1)<  ^  ,  t  >  O. 

L  y  € 

Here  x,y  are  scalar  state  variables  amd  scalar  control  variables,  con¬ 

strained  by 

2 

a  given  closed  subset  of  R  .  Motivations  for  considering  system  (i)  come  from  i^obo 
tics,  since  some  dynamic  equations  of  kinematic  chains  appearing  in  robotics  may  be 
reduced  to  the  form  (1).  See  e.g.  for  a  recent  treatment. 

The  multifunction  which  describes  the  unluiown  system  dynauaics  is  given  by 

4  2 

(Gj,  63)  '  :  [O,  +  00)  X  R  X  U  r?  R  . 

We  assume  explicit  knowledge  of  some  upper  and  lower  bounds  of  the  dynamics  involved. 
Therefore  CarathSodory  functions 

,  i  =  1,2 

are  known  such  that 

G)  =  [g;  ,  g^]  .  G^  =  [g^  ,  g^  ]  . 

The  initial  state  is  uncertain  but  bounded  by  some  known  constant. 

We  consider  a  given  linear  time  invariant  (known)  model 

(2)  *  =  a^w  t  a^w  +  a^v^  ,  Z  =  b^i  +  b^z  +  b3V2 

with  scalar  control  variables  v^  v^  ,  state  varied>les  w  ,  z,  w(0)  anu  z(0)  fixed, 
and  arbitrarily  fixed  coefficients  a^,  b^^  (i  -  1,...,3). 

The  problem  we  consider  is  to  find  a  state  feedback  control  law  (possibly  depen 
ding  on  instantaneous  values  of  v^,  v^,  w,  w  and  z,z)  under  which  system  (1)  is  asym 
totically  equivalent  to  the  given  linear  model  (2) . 
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Mora  precisely,  given  0<  >  o  emd  any  (v^jV^rW.z) ,  we  construct  (in  a  sense  ex- 
plicitely)  a  feedback  u  such  that  every  possible  state  (x,y)  for'  (1)  corresponding 
Co  it  (under  any  uncertain  dynaaics  g  ^  (G^,  G^)')  fulfils  the  oodel  dynamics  (2)  up 
to  an  exponentially  decaying  error  term  dominated  by  (const.)  exp  (  -0<t) ,  t  )  T; 
moreover 

lx  (t)  -  V/(t))  ^  (const.)  exp(-o(  t) ,  t  >  T 

where 

X  •  (X,  X,  y,  y)  «  (x^,  x^,  x^,  x^) ,  W  »  (w,  w,  z,  i)  -  <yi'y2'  ^3'  ^4*' 

and  some  T  independent  of  X  and  explicitely  estimated  by  known  data. 

Such  a  feedback  u  (in  general  discontinuous)  is  obtained  by  using  veuriedsle  stru£ 

Cure  control  methods  (see  recently  extended  to  non  linear  control  systems  by  the 

authors  (see  C2]  and  fsj  ) ,  provided  a  set  of  explicit  inequalities  is  satisfied 
+ 

by  the  (mown  bounds  g~  ,  i“  1,2  ,  as  follows.  Let  c  ,  c  such  that  o<^  min(c  ,c.)  . 

i  12  12 

Then  put  e  =  X  -  M  and 

(3)  s^(e)  =  e^  >  e^  ,  (e)  *  *  e^. 

Consider  now 

p(x,  y,  V)  -  Cj  (y^  -  Xj)  ♦  yj  +  ^1  *  “3  ''l' 

q(x,  y,  V)  «  Cj  (y^  -  x^)  +  tjj  y^  ^2  ^3  *  ^3  '^2' 

Then  we  assume  existence  of  some  u  C  U  fulfilling 

g‘  >  p  +  k^  if  Sj  >  O,  g*  «  p  -  k^  if  Sj  <  0, 


r  1 
’{  - 

I  g,  >  q  + 


2  +2 

k  if  s^  >  O;  g^  4^  q  "  k  if  s^  <  0. 


These  results  may  be  generalized  to  higher-order  control  systems  of  the  following 


X  €  G,  (tf  X  ,  X  ,  ...,  X  ,  X  ,  u  ,...,  u  ),  i  ■  l,...,n. 
ii  11  nnl  n 

It  is  then  likely  that  the  number  of  the  required  inequalities  corresponding  to  (4) 

may  be  reduced  by  using  results  of  CO  . 

The  chattering  effects  due  to  the  discontinuous  nature  of  the  asymptotically 

linearizing  feedback  may  be  reduced  by  appropriately  combining  results  from  [8]]  , 

CO  ,  [icj  .  Moreover  sufficient  conditions  nay  be  obtained  assuring  that  the  feedback 

u  is  piecewise  continuous  (not  only  measurable) . 

An  important  property  of  the  variable  structure  control  systems  (1),  (3)  is  a£ 

proxlmabillty  (see  CO  the  precise  definition) . 


57 


In  essence  this  means  that  whenever  sooie  error  vectors  e^  depend  on  disturbances 
described  by  the  real  parameters  £  and  satisfy 


Si  (eg  ) 


0  as  £ 


O,  i  »  1,2, 


uniformly  on  coiq>act  intervals  of  Tt,  ^} ,  then  e.  - >  e  in  the  same  sense ,  where 

c  o 

e  is  uniquely  defined  and  fulfils  the  sliding  condition  s  (e  )  •  0,1  »  1,2.  Thus 
o  1  o 

approxlmedsillty  prevents  ambiguous  behaviour  in  the  sliding  mode.  Sufficient  cond^ 
tions  can  be  fovmd  about  the  available  data  in  order  to  fulfil  such  a  property. 

A  PARTICULAR  CASE.  Let  the  uncertain  control  system  be  described  by  a  single  diff£ 
rential  inclusion  of  order  n  with  scalar  control  u,  given  by 


*2  '  *2 


seer  X  *  X  P  X  ^  G(t,  X,  u)  f  u  £  U. 
3  n-1  n  n 


Suppose  that  the  model  is  given  by 

n 

^  =  ^2  ’  ^2  =  ^3 . Vl  =  ^n'  ^n  '  S  Vi  " 


=  [g~,  g*J 


Fix  real  numb'sr  c  ,...,c  such  that  the  polynomial  (in  h) 
1  n-1 


.n-1  .n-2 

h  +  c  ,  h  ♦ 
n-1 


+  c  h  +  c, 
2  1 


is  Hurwitz.  Denote  by 


e  =  y  -  X 


the  error  vector  and  set 


s(e)  *6 


*— r  '=  ®- 

1-1  1  1  , 


p(x,  y,  v)  •  bv  +  >  ay  +  c  (y  -  x  )  . 

i»l  ri  i»l  i  i+l  i+1 

The  asymptotic  linear  behaviour  (  in  the  sense  defined  above)  obtains  provided  the 
following  holds.  For  any  model  control-state  pair  y,v,  every  t  >  0  and  every  x  we 


can  find  u  €  U  such  that 


g  (t,  X,  u)  >  )c  +  p(x,  y,  v)  if  s(y  -  x)  >0, 

+  2 

g  (t,  X,  u)  <  -  k  ♦  p(x,  y,  v)  if  s(y  -  x)  <0, 


for  some  fixed  constant  k  O. 

Arbitrary  exponential  decay  of  the  error  term  is  obtained  by  a  proper  choice  of 
Cj,  ...,c^  j  .  See  H  ■  for  snore  details,  examples  and  coopeurisons  about  this 
particular  case. 
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ABSTRACT 

This  paper  analyzes  the  performance  of  output  feedback  controllers  for  a  class  of 
uncertain  time-varying  nonlinear  systems  in  the  presence  of  unmodeled  actuator  and 
sensor  dynamics.  In  particular,  on  the  basis  of  known  nominal  model  and  bounds  on 
the  uncertainties,  and  initially  neglecting  actuator  and  sensor  dynamics,  high-gain  out¬ 
put  feedback  schemes  are  determined  which  force  the  output  to  track  a  given  signal. 
Then,  the  effects  of  actuator  and  sensor  dynamics  are  investigated  on  the  performance 
of  the  tracking  system. 

KEY  WORDS*  Nonlinear  systems,  Output  feedback.  Uncertain  systems.  Singular 
perturbations. 

1.  INTRODUCTION 

Recently,  major  progress  has  been  made  in  the  analysis  and  design  of  nonlinear 
control  systems.  Different  approaches  have  been  proposed  (Utkin,  [l],  [2j,  Corless  and 
Leitmann  (17|,  Hunt  et  al.  [5j,  Su  et  al.  [6|,  Glad  [22],  [23],  Bauman  and  Rugh  [lOl, 
DeCarlo  et  al.  [10],  Isidori  [iSj,  Walcott  and  Zak  [8],  Steinberg  and  Corless  [12]).  An 
important  property  of  control  systems  is  their  robustness,  i.e.  the  ability  of  the  system 
to  retain  certain  performance  measures  in  the  presence  of  perturbations.  Or  in  other 
words;  “the  ability  of  a  control  system  to  function  even  when  the  actual  system  differs 
from  the  model  used  for  designing  the  controller’’  (Glad  [22]).  The  system  model  used 
by  the  designer  may  differ  from  the  controlled  system  because  of  model  uncertainties  or 
neglected  high-frequency  dynamics.  Specifically,  when  devising  a  model  of  the  plant, 
small  time  constants  corresponding  to  actuator  and/or  sensor  dynamics  are  neglected. 
Furthermore,  it  is  often  impossible  to  measure  directly  all  the  components  of  the  state 
or  output  vectors.  In  order  to  restore  them  additional  sensors  are  used  which  lead  to 
motions  different  from  the  motions  predicted  by  the  plant  model. 


The  problem  of  eontroUlng  e  iTitem  in  the  preicnee  of  unmodeled  eetuetor  end 
sensor  dynamics  has  ree«v«l  recently  the  attention  of  many  researchers.  In  particular 
Bondarev  et  al.  [7],  and  Zak  et  al.  [2Sj  studied  the  influence  of  neglected  high>frequency 
dynamics  on  the  variable  structure  control  systems.  Leitmann  et  al.  [9j  studied  the 
robustness  with  respect  to  neglected  actuator  and  sensor  dynamics  of  state  feedback 
controllers  for  uncertain  systems.  Glad  [23]  considered  the  sen^tivity  of  the  system  to 
variations  in  gain  at  the  input,  corresponding  to  nonideal  behavior  of  the  actuators. 
The  problem  of  the  robustness  of  various  output  feedback  control  algorithms  based  on  a 
reduced*order  model  with  neglected  high-frequency  dynamics  wsa  investigated  by 
O’Reilly  [18]  and  Vostrikov  et  al.  [24]  using  singular  perturbation  techniques. 

The  purpose  of  this  paper  is  to  analyze  the  effect  of  neglected*  high-frequency 
dynamics  on  various  output  feedback  control  designs  for  nonlinear  uncertain  systems. 
Our  approach  is  inspired  by  Marino  [4],  Utkin  [2],  and  Vostrikov  et  al.  [24j.  The  tools 
we  use  in  this  paper  are  the  high-gain  output  feedback  and  Lie  derivatives. 

The  paper  is  organised  as  follows.  Section  2  is  devoted  to  the  description  of  the 
class  of  nonlinear  systems  we  consider  along  with  the  problem  statement.  The  next  sec¬ 
tion  presents  some  background  material  and  preliminary  results.  The  following  sections 
discuss  different  high-gain  output  feedback  control  schemes.  Then  the  effects  on  the 
performsmce  of  the  closed-loop  system  of  unmodeled  actuator  and  sensor  dyntmics  are 
investigated.  Finally,  Section  6  contains  concluding  remarks. 

2.  PROBLEM  STATEMENT 

In  this  paper  we  consider  a  class  of  dynamical  systems  governed  by  the  following 
equations 

x(t)  -  f(t,x)  +  G(t,x)  [u(t)  -I-  f(t,x)| 
y(t)  -  h{x) , 

where  xGlR®,  uGlR”,  y€]R“,  and  f]*)  RxR®  — *51“  is  the  lumped  uncertain  element. 
We  assume  that  the  norm  of  the  uncertain  element  is  bounded  by  a  known  bounded 
nonnegative  function;  that  is  for  all  (t,x)€lRxIR‘ 

li^(t,x)|i  £  p(t,x) , 

A 

where  p(*) :  IRxlR"— 1R+,  and  ll*||  is  the  Euclidean  norm  i.e.,  ||x|(  (2  l*i  I*)*'^*- 

i-l 

Note  that  the  only  information  assumed  about  the  uncertain  vector  is  its  maximum 
potstbie  energy.  If  the  uncertainties  ^(t,x)  enter  structurally  into  the  state  equations  as 
in  (2.1)  then  we  say  that  the  matching  condition  ’is  satisfied  [17|. 

The  function  f(«]  is  a  continuous  single-valued  vector-fhnetion  and  G(*)  is  a  con¬ 
tinuous  single-valued  matrix  ^lnction  with  rank  G  m.  Furthermore,  we  require  that 
f(t,0)  0  for  all  t.  The  output  vector  function  h(*)  is  continuously  differentiable  and 


h(0)  -  0. 

In  this  paper  we  analyze  two  different  output  feedback  control  strategies.  Tbe  first 
is  the  high-gain  output  feedback  stabilization  scheme.  In  the  synthesis  of  this  control 
law  we  utilise  a  nonlinear  transformation  wMch  brings  the  original  system  into  the 
“regular  form"  ([20]}  from  where  the  design  is  performed. 

The  aim  of  the  second  control  law  is  to  ensure  the  tracking  property  of  the  output 
of  some  given  reference  signal. 

For  both  control  strategies  we  will  investigate  the  effects  of  the  unmodeled  actuator 
and  sensor  dynamics  on  the  performance  of  the  closed-loop  systems. 

3.  PREUMINARY  REStJLTS 
LIE  derivatives 
Time-Invariant  Lie  Derivatives 

Let  f  :  ]R“— IR*  and  g  :  ]R“— •E.®  be  C“  vector  fields  on  The  Lie  bracket  is 
defined  by 

!f,g]  ^  ^  > 

df 

where  and  are  the  Jacobian  matrices  of  f  and  g,  respectively.  Using  an  altema- 
tive  notation,  one  can  represent  the  Lie  bracket  as  follows 

[f.g]  -  (ad*f,g) . 

Also,  define 

(ad'‘f.g)  -  if,(ad‘'-‘f,g)l  , 

where,  by  definition 

(ad®f,g)  -  g  . 

Next,  consider  a  C“  function  h  :  R“— *R.  Let  dh  "  V^h  be  the  derivative  of  h 
with  respect  to  x,  where  Vh  is  the  gradient  of  h  with  respect  to  x.  Then  the  Lie  deriva¬ 
tive  of  h  with  respect  to  f  is  defined  by 

Lfh  -  L,(h)  -  <dh,f>  -  V^h-f . 

The  following  not&tion  is  employed  throughout  this  peper 
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1^ 


% 


L?h  -h 

yh-Lfb 


Ljfli-L,{Lh‘h) . 


The  Lie  derivative  of  dh  'with  respect  to  the  vector  field  f  is  defined  by 


L,(dh) 


One  may  easily  verify  that  these  three  Lie  derivatives  obey  the  follo'wing  so-called  Leib¬ 
nitz  formula 


L|f  fjh  “  <dh,[f<gj>  •  LfLfh  —  LfLjh  . 
Furthermore,  the  following  relation  is  valid 

dLfh  -  Lf{dh)  . 


Time-Varying  Lie  Derivatives 

Suppose  now  f  and  g  are  C*  time- varying  vector  fields,  i.e.  f(*)  :  IRxlR”— R®, 
g(*)  :  IRxlR®— »1R“.  Then  the  time- varying  Lie  bracket  is  defined  by 

(r‘f,g)^(ad‘f,g)--|-, 

and 

(r''f,g)-(r'f,(rk-‘f,g)), 

where 

Next  consider  a  0“  function  h(*)  :  IRxlR®— »IR.  Then  the  time- varying  Lie  derivative 
of  h  with  respect  to  f  is  defined  by 

if,h-i?,(h)^L,h-(--^. 

We  define 


jf?h^h  , 

I, 

ifl^h  -L,(if?-»h)  . 

The  tixDft-varyms  Lie  derivecive  of  dh  with  respect  to  the  time-verying  vector  field  f  is 


/ 


defined  by 

i?,dii -fT  +(dh) ^  +  A (dh) . 

Note  that 

di?fh  -i?r(dh). 

One  may  verify  that  the  above  defined  time-varying  Lie  derivatives  obey  the  following 
formula 

<dh,(r‘f,g)> 

“  L(r>f,j)h  ™  LjLfh  —  LfL^h  —  L.|t.h 
-  Ljiffh  -  iffLjh  . 

MARKOV  PARAMETERS 

The  affine  Markov  parameters  are  defined  as  the  elements  of  the  matrix  resulting 
from  the  product  of  the  observability  and  controllability  matrices  c'  an  affine  nonlinear 
system  described  by  the  following  equations 

X  -  f(t,x)  +  gi(t,x)u,  +  ...  +  gm(t.x)Um 
y  -  h(x)  -  fhi(x) ,  ... ,  hp(x)p  , 

where  f,gi,..-,goi  ■  IRxIR“— *11“  and  h  ;  1R“— *IR’’  are  C”  vector  fields. 

The  observability  matrix  of  such  a  system  is  defined  by  the  following  (np)xn 
matrix 


6S 


Continuing  in  this  manner  we  &nd  that 

—  const  —  q+j  . 

□ 

For  further  information  about  Markov  parameters  for  nonlinear  time-invariant  sys¬ 
tems  the  reader  is  referred  to  [ll]  and  [14]. 

Consider  now  a  plant  modeled  by  (3.1),  where  p  •  m,  and  the  high  gain  control 

law 

u  -  k  s(x)  (3.6) 

where  k  >  0  is  a  scalar  and  the  function  s(*)  :  R”— *1R“  is  continuously  differentiable. 
Assume  that  detSG  ^  0,  where 

ds 

S--^,  and  G  =  [gi,g2,...,ga,j  . 

Then  we  have 

Theorem  3.2  (!2';,;2l!); 

If 

(i)  the  functions  f(t,x),  G(t,x)s(x),  and  f<,  —  f  —  G(SG)“*Sf  satisfy  Lipschit*  condi¬ 
tions  for  all  X 

(ii)  the  system 

is  uniformly  exponentially  stable,  that  is  there  exist  positive  A  ^  1  and  at  such 
that 

||s(x)ll  <  A||s(x(0))|le-“‘  , 

then  for  any  positive  A,  and  T  there  exists  a  positive  ko  such  that 

||s(x(t))||  <  A 

for  k  >  ko  and  to  -i-  tj  <  t  <  T  on  the  solutions  of  (3.1)  with  the  control 
u  *  ks(x),  and  lim  ti  »  0. 
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4.  THE  OUTPUT  REGULATION  PROBLEM 

Consider  the  nominal  system,  that  is  the  system  without  uncertainty  as  described 
by 

i  —  f(t,x)  +  G{t,x)u 
y  h(x) . 

First  we  define  the  decoupling  indices  for  the  system  (4.1).  We  consider  each  of  the 
m  output  channels  separately.  So  considering  the  first  output  channel  we  form  the  fol¬ 
lowing  row  vector  which  we  will  call  the  decoupling  vector  for  channel  one 

!L,,hi,  L,,ht,...,Lj^hi]  .  (4.2) 

If  this  row  vector  is  not  identically  equal  to  zero,  then  we  define  the  decoupling  index  of 
the  first  channel  to  be  zero,  or  »  0. 

However,  if  the  row  vector  is  identically  equal  to  zero  we  proceed  to  form  the  fol¬ 
lowing  decoupling  vector 

m.iffhi,  . L,.if,h,i . 

Again  we  determine  if  it  is  identically  equal  to  zero,  or  not.  If  it  is  not  we  stop  and 
define  di  =  1.  If  it  is  zero  we  proceed  further  by  forming 

:L,,if?hi.  ,  ... ,  , 

and  so  on. 

So  the  decoupling  index  of  channel  1,  is  equal  to  the  smallest  integer  dj  for  which 
the  decoupling  vector, 

[L,.i??'h,.Lj,i??'hi,...,L,„iff'h,i, 
is  not  identically  equal  to  zero. 

Similar  procedure  for  the  other  output  channels  yields  a  set  of  m  parameters,  dj  for 
i  >■  l,2,...m. 

The  decoupling  indices  are  an  indication  of  what  the  lowest  derivative  of  each  out¬ 
put  channel  needs  to  be  utilized  for  an  output  control  to  be  effective.  By  taking  the 
time  derivative  of  the  i^  output  channel  we  obtain 

3hi  .  ahi 

Xi  -  (f  +  *i«i  +  -  +  . 

hence 

yj  ™  Lfhj  -1-  ,  ••• ,  u  . 

Thus  if  iL(,bi,  Lg,hi  ,  ... ,  L,^hiJ  w  |0j  then  u  has  no  effect  on  the  output  yi,  so  we 
need  to*form  yj  where 
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y-.  -  . 

Again  if  [L(,iffhi,  Lf,iffhi  ,  ... ,  »  |0j,  then  u  has  no  effect  on  the  output 

and  we  need  to  take  higher  derivatives  of  in  a  similar  fashion  as  before. 

Now  that  we  have  obtained  the  set  of  decoupling  indices,  we  consider  all  the  out¬ 
put  channels  together  to  form  the  following  matrix 


We  will  assume  that  the  matrix  N  is  nonsingular  and  we  will  further  assume  that 
the  Markov  parameters  of  the  system  (4.1)  are  constant.  Hence  by  the  virtue  of 
Theorem  3.1  the  matrix  N  is  constant. 

With  the  N  matrix  constant  and  nonsingular,  we  proceed  to  construct  a  high-gain 
output  control  which  will  regulate  the  output  to  zero. 

We  will  consider  two  cases.  The  first  case  is  when  all  decoupling  indices  are  equal 
to  zero,  and  the  second  case  when  some,  or  all,  decoupling  indices  are  not  equal  to  zero. 

For  a  rigorous  treatment  of  the  decoupling  problem  for  nonlinear  time-invariant 
systems  the  reader  is  referred  to  I3t,  [l4j,  [18|. 

Case  1:  For  this  case  the  N  matrix  will  have  the  following  form 


Ljjha,  ...  Lj„hn 


#-G=HG. 

ax 


where  is  the  Jacobian  matrix  of  h  and  G  “  (gi,— <gmi' 

Ox 

If  we  employ  the  following  diffeomorphic  state  variable  transformation 


X,  -  <>i{t,x) 

X2  -  ^(t,x) 

Xn-m+l  “b,(x) 

Xa  -  hn,(x)  , 

where  the  O-,  's  are  chosen  such  that 

Lj  Oi  =■  0  ,  j  =  for  ail  i  =  m, 

then  the  system  (2.1)  in  the  new  coordinates  will  have  the  following  form 

xi  iffOi 

iffOa-m 


X  1 


L,h, 

Lj.h,  .. 

Ufh„ 

L,,bm  - 

(u  -r  0  • 


We  will  aow  employ  the  high  gain  output  feedback  control  as  given  by 

u  =  y  K'’h(x)  ,  (4.3) 

where  t  is  a  small  constant  and  K°  is  an  mxm  constant  matrix.  Under  the  influence  of 
this  control,  the  system  equations  become 


*11— m+1 


Lfh, 

Lfh„ 


+  j  N  K®h{x)  +  . 


(4.9b) 


We  see  that  the  application  of  this  control  decouples  the  system  into  the  slow  and 


fast  subsystems.  The  dynamics  of  (the  slq 

*1 


|W  subsystem  are  given  by 


y  -  h(x) 


m+1 


0, 


1 

«  .N  K®h(x)  -  N  K® 

^11— CD'*'! 

(4.10) 


whereas  by  invoking  the  following  change  in  the  time  variable, 

t=€r, 

the  equations  c.  scribing  the  dynamics  of  the  fast  system  are  given  by 
fxo  — 01  +  1  ^f^l 

I  ;  -  €  :  +  N  K'’h(x)  +  , 

I  Xn 

and  for  sufficiently  small  t,  the  above  equations  simplify  to 


dr 


Observing  that  the  part  of  our  transformation  in  (4.5)  is  y  ■  [xa-m-i  ’ 
we  can  rewrite  the  above  equation  as 

^-NK-y. 


(4.11) 


(4.12) 


(4.13) 


(4.14) 


Note  that  by  an  appropriate  choice  of  the  matrix  K®  the  fast  subsystem  can  be 
made  uniformly  exponentially  stable.  Thus  if  J  is  the  required  uniformly  exponentially 
stable  matrix  then, 


(4.15) 


K®  -N->J, 

and  K°  can  be  evaluated  since  N  is  assumed  to  be  nonsingular. 

By  invoking  Theorem  3.2  we  see  that  the  stability  of  the  fast  subsystem  will  result 
in  the  trajectories  of  the  system  (4.7)  converging  to  the  A-vieinity  of  the  manifold 
y(x)  ~  0.  Thus  the  output  is  regulated  to  zero.  Within  the  A-vicinity  of  the  manifold, 
the  system  will  be  governed  by  equation  (4.10)  which  represents  the  dynamics  of  the 
slow  subsystem.  From  equation  (4.10),  we  notice  that  we  do  not  have  any  influence  on 
the  internal  stability  of  the  slow  subsystem  when  the  output  is  regulated  to  zero.  We 
assume  however  that  the  slow  subsystem  is  asymptotically  stable.  The  stability  of  the 
stow  subsystem  is  a  structural  property  of  the  plant.  This  subject  requires  further 
research. 

Although  Theorem  3.2  was  stated  for  nonlinear  systems  without  uncertainties,  it 
also  applies  to  our  particular  case.  This  is  because  the  uncertainties  in  the  system  (2.1) 
are  bounded  by  a  known  bounded  function. 

Case  2i  Let  us  first  reorder  the  output  channels  so  that  they  are  ordered  in  ascending 
values  of  their  decoupling  indices.  Thus  y^  is  assigned  to  the  channel  with  the  smallest 
di,  and  ym  to  the  one  with  the  largest  di. 

We  then  employ  the  following  difleomorphic  state  variable  transformation 

Oi(t,x) 

Oj(t,x) 

lim(x) 

,  (4.16) 

iff'h, 

.  . 

where  the  o,  's  are  chosen  such  that 

*0  j  w  l,...,m  . 

The  system  (2.1)  in  the  new  coordinates  will  have  the  following  form 
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£*  -  f,(t,x)  +  N(u  +  0  . 


(4.17) 


where  3C*€1R“,  and  N  is  given  by  (4.3).  The  existence  conditions  of  the 

transformation  (4. IS)  can  be  deduced  from  the  results  of  [5j,  [20],  [26],  [27]. 

Note  that 


*i 


,  i  “  l,...,m  , 


where  (  )(!'  denotes  the  j-th  derivative  of  (  )  with  respect  to  t.  The  control  law  will  have 
the  form 


u 


Iti.iri  +•••  + 

I 

^m.tym  •••  ^m,d„Tiyin 


(4.18) 


In  the  new  coordinates  the  closed-loop  system  (4.17),  (4.18)  is  decoupled  into  the  slow 
and  fast  subsystems.  The  slow  subsystem  is  governed  by  the  equations 


x'  =  fi(t,x) 
y  »  0  . 


(4.19) 


As  in  the  previous  case,  we  have  no  influence  on  the  stability  of  the  slow  subsystem. 
Therefore  for  the  controller  to  be  effective  we  have  to  assiune  that  the  system  (2.1) 
without  uncertainties  is  asymptotically  stable  when  restricted  to  the  manifold  y  ^0 
which  is  equivalent  to  requirement  that  the  system  (4.19)  is  asymptotically  stable. 

.\s  with  regard  to  the  fast  subsystem  we  utilize  a  change  in  the  time  variable 
t  =  «  r  to  obtain 


(4.20) 


-  «(fi  +  NO  + 

^rn.iym  **•  ^m,d„+iyin 

If  we  now  choose  k,j  in  such  a  way  that  the  simplified  fast  subsystem 
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Ici.iyi  +  - +lci,i,+iy^i‘‘'' 

: 

: 

ia  uniformly  exponentially  stable  then  by  the  virtue  of  Theorem  (3.1)  the  closed-loop 
system  is  asymptotically  stable. 

The  above  output  feedback  stabilization  schemes  are  quite  restrictive.  Their 
efiectiveness  depends  on  the  stability  of  the  nominal  systepi  (x  •  f  -f  Gu)  when  res¬ 
tricted  to  the  manifold  y  —  h(x)  —  0.  In  the  followiue  section  we  provide  a  more 
effective  control  scheme.  Before  that  however,  we  will  analyze  the  effect  of  unmodeled 
actuator  dynamics  on  the  performance  of  the  closed-loop  system  with  the  high-gain  out¬ 
put  feedbacks. 

SYSTEMS  WITH  FAST  UNMODELED  MOTIONS 

We  now  investigate  the  effects  of  the  introduction  of  uncertain  actuator  dynamics 
on  the  performance  of  the  system  (2.1)  with  high  gain  output  feedback  controllers. 

Case  1;  We  will  assume  that  the  actuator  dynamics  is  modeled  by  the  following  equa¬ 
tion 

=•  Lr  -I-  Mu  ,  u  =«  Nr  -f  Ru  ,  u  —  c  K'y  ,  (4-21) 

where  r^lR'^,  q  ^  m,  L  is  a  Hurwitz  matrix,  is  a  positive  constant  that  reflects  the 
“fastness"  of  the  actuator,  the  matrices  L,  M,  R,  and  N  satisfy  the  condition 

R  —  NX'^M  =»  In,,  and  c  —  —  is  a  large  constant. 

Proposition  4.1:  If  the  matrix  L  is  Hurwitz,  (the  fast  subsystem  describee  by  (4.21)  is 
exponentially  stable)  then  as  approaches  0,  the  motion  of  the  slow  subsystem  is 
described  by  (2.1)  with  u  »  u  =•  c  K^y. 

Proof;  The  fast  subsystem  is  described  by  (4.21).  Replacing  u  by  its  value  yields 

••  Lr  -i-  cMK’y  .  (4.22) 

Let  T  ^j*t,  hence  (4.22)  becomes 

-  Lr  -H  cMK'y  .  (4.23) 

qt 

Since  L  is  a  Hurwitz  matrix,  then  as  r  approaches  infinity  we  have  y  «  constant  and 


f 


lim  r  »  —  cL~*MK®y  , 

f— 00 

hence 

u-Nf +  cRIC’y-(-NL“‘M+RIcK“y-cK®y  .  (4.24) 

The  expression  for  u  as  per  (4.24)  can  also  be  found  by  setting  iJ^  •^Q.  Hence  the  slow 
subsystem  is  described  by  (2.1)  and  (4.24). 

□ 

Case  2:  If  the  actuator  dynamics  for  this  case  is  also  described  by  (4.21)  with 

Ki.iy,  +  —  + 

3  =  cN”*  • 

then  using  a  similar  argument  as  in  the  previous  ease  we  conclude  that  the  slow  subsys¬ 
tem  is  described  by  (2.1)  with  u  »=  u. 

In  conclusion,  for  a  sufficiently  fast  actuator  the  proposed  control  schemes  will  sta¬ 
bilize  the  output. 

5.  THE  TRACKING  PROBLEM 

Our  goal  now  is  to  design  a  controller  such  that  the  output  of  the  system  (2.1)  will 
track  a  given  reference  signal. 

A  sufficient  condition  for  the  output  y  to  track  the  reference  signal  iA,i)  is 

^  iy  -  ^t)]  =  Vly  -  t<t)i  ^  F(y,.4t))  -  (5.1) 

where  V  is  a  Hurwitz  matrix.  If  t^t)  =  constant,  then  (5.1)  becomes  y  “  V[y  —  i/]. 

We  require  that  the  closed-loop  system  (2.1)  be  asymptotically  stable  with  respect 
to  the  time-varying  manifold 

n  -  {x  :  h(x(t))  -  lit)  -  y(t)  -  i^t)  -  0}  • 

The  projection  of  the  overall  system  on  this  manifold  is 
Ht)  -  i<t)  -  Hx  -  lit) 

-  Hf  -t-  HG{u  +0-iit). 

Using  equation  (5.1)  and  solving  for  u,  we  obtain  the  following  control  law 

3  -  (HG)-‘  IF(y,  .^t))  -  Hf  +  i^t)]  -  ^  (5.2) 

In  order  to  Implement  the  control  law  (5.2),  we  would  have  to  have  the  exact  knowledge 


of  the  uncerUin  vector  ^(t,x).  Hence  this  control  strategy  is  impractical.  In  what  fol¬ 
lows  we  propose  a  practical  control  algorithm  which  approximates  the  controller  (5.2). 
Consider  the  following  control  strategy 

u  -  K[V(y  -  t<t))  -  (y  -  i<t))]  ,  (5.3) 

where  K  is  the  matrix  of  gain  coefficients,  K  eK°,  and  c  is  a  scalar  large  factor.  At 
the  present  time,  we  will  assume  that  y  can  be  measured  exactly.  Later,  we  will  investi¬ 
gate  the  ease  in  which  y  is  measured  by  a  sensor. 

To  analyze  the  behavior  of  the  system  (2.1)  with  the  control  law  (5.3)  in  the  pres¬ 
ence  of  unmodeled  actuator  and  sensor  dynamics  we  will  employ  the  arguments  of  Vos¬ 
trikov  et  al.  [24|  used  for  systems  without  uncertainties. 

Along  the  trajectories  of  the  motion  of  the  dynamical  system  (2.1),  y  is  given  by 

y  -  Hf(t,x)  +  HG(t,x)(u  -(-  ^(t.x))  .  (5.4) 

Propc'^ition  5.1:  If  det(I  -f  cHGK*’)  #  0,  and  det(HG)  ^  0,  then 

(a)  Itm  [y  -  bit)]  =  F(y,i^t)), 

c-*oo  al 

(b)  lim  u  -  (HG)-'  lF(y,^(t))  -  Hf  ^  ^t)  -  HG^!- 

Cw^OO 

Proof;  In  what  follows  we  shall  utilize  the  arguments  of  V'ostrikov  et  al.  [24l. 

We  first  prove  pan  (a).  Recall  that 

y  -Hi  ,  i  -  f  ^  G(K1F  -  (y  -  u)\  +  0  , 

thus,  we  have 

y  -  Hf  +  HGK(F  -  y  -.1)  HG^  , 
regrouping  the  y  terms  leads  to 

(I  -r  HGK)y  -  Hf  +  HGK(F  +  u)  +  HG^  , 

Hence,  for  K  —  cK®,  we  have 

y  -  (I  +  cHGK'’)-‘(Hf  +  HGO  +  (I  +  cHGK®)-'cHGK®(F  -I-  u)  . 

Taking  lim  y,  the  first  term  approaches  zero,  while  the  second  term  approaches  F  4- 

C-^OO 

therefore 

lim  y  —  F  +  i/  . 


We  now  prove  part  (b). 


We  heve 

u-K[F-{y-^)j,  y-Hf+HG(u+0. 

therefore 

u-K(F -Hf-HG(u  +  0  +  i^  • 

Regrouping  the  u  term)  leads  to 


hence,  for  K 

Thus 


(I+KHG)u-K[F-Hf-HG€  +  £/]  , 
cK®,  we  have 

u  -  (I  +  cK®HGr'cK®  [F  -  Hf  -  HG^  +  . 


lim  u  -  {HG)-‘  [F  -  Hf  -  HGf  +  u]  . 


□ 


SYSTEMS  WITH  FAST  UNMODELED  MOTIONS 

We  will  now  investigate  the  effects  of  the  neglected  actuator  dynamics  on  the  per¬ 
formance  of  the  system  (2.1)  with  the  control  law  (5.3). 

Suppose  that  the  actuator  dynamics  is  modeled  by  the  following  equation 

M»r  —  Lr  -i-  Mu  ,  u  Nr  ,  u  —  K(F  —  y  +  t')  ,  (5.5) 

where  r€H'’,  q  ^  m,  L  is  a  Hurwitz  matrix,  ju,  is  a  positive  constant  that  reflects  the 
"fastness"  of  these  dynamics,  and  the  matrices  L,  M,  N  satisfy  the  condition 
-NL‘‘M-I. 

The  system  described  by  (2.1),  and  (5.5)  may  be  studied  by  the  methods  of  the 
theory  of  differential  equations  with  small  parameters  in  some  of  the  derivatives  [24]. 

For  such  systems,  the  overall  motion  can  be  decoupled  into  the  fast  and  slow  com¬ 
ponents  [2ll  [24].  The  method  of  decoupling  motions  is  advantageous  in  systems  involv¬ 
ing  high-gain  feedback  and/or  singular  perturbations.  The  main  idea  behind  the  theory 
is  to  decouple  the  system  into  two  subsystems  of  lower  dimensionality.  The  equations 
of  the  slow  motions  and  the  convergence  conditions  for  the  fast  motions  are  examined  in 
[211  and  |24i. 

In  the  following  proposition  we  investigate  the  effects  of  the  actuator  dynamics  on 
the  performance  of  the  system  (2.1). 


Proposition  6 .2:  If  the  mstrix  (L  —  MKHGN)  is  a  Hurwitz  matrix,  then  as  c— *oc  the 
motion  of  the  slow  subsystem  will  be  described  by  (2.1)  with  u  u. 


Proof:  As  the  slow  subsystem  is  described  by  the  following  equations 

X  «  f  +  G(u  +  ?) ,  u  *  u  —  K(F.  —  y  +  t') . 

We  now  examine  the  condition  for  the  stability  of  the  fast  subsystem. 

The  fast  subsystem  is  described  by  equation  (5.5).  Replacing  u  by  its  value  yields 
-  Lr  +  MK(P  -  Hf  -  HGNr  -  HG^  +  i/) .  (5.6) 

Let  r  —  hence  equation  (5.8)  becomes 

=  (L  -  MKHGN)r  +  MK(F  -  Hf  -  HG^  +  iA  .  (5.7) 

aT 

where  x  »  constant,  t  constant. 

If  the  matrix  (L  —  MKHGN)  is  Hurwitz,  then 

lim  r  =  -  (L  -  MKHGN)-‘MK(F  -  Hf  -  HG^  +  i/j  . 

r-woc 

Applying  twice  the  following  matrix  identity  know  as  the  matrix  inversion  iesuna 
(All  +  AijAjj.A.;i)"*  “An  —  AJ'/Ai2(AjiAii*Ai2  -i- -AJi*  )~^.A2i.A7i’  . 
and  the  condition  —  NL"‘M  =  la  we  obtain 
lim  N(L -MK*(cHG)N)~‘c.MK'’ 

C— 

-  lim  cK'^'-Ia  -r(K'’  -(cHG)-')“'K‘’! 

=.  -  (HG)-*  .  (5.8) 

Hence 

lim  u  w  lim  Nr  =•  (HG)“'[F  —  Hf  +  t-i  —  ^  . 

C— •*  C— *00 

□ 


INFLUENCE  OF  SENSOR  DYNAMICS 

To  implement  the  control  law  (5.3),  the  vector  y  has  to  be  measured  by  a  sensor 
(approximate  difierentiator).  Suppose  that  the  approximate  differentiator  is  modeled  by 
the  following  equation 
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IXt*  ■*  A*  +  Dh(x] 
y-P». 

where  x€IR‘>,  y€lR°‘,  q  2:  m,  and  is  a  '‘smaH”  parameter  that  reflects  the  "fastness" 
of  the  approximate  differentiator,  y  is  the  estimate  of  y,  A  is  a  Hurwitz  matrix,  and  the 

matrices  P,  A,  and  D  satisfy  the  condition  —  PA~*D  »  L  We  shall  also  use  y  instead  of 
y  in  the  control  law  (S.3).  Therefore,  we  have 

y  -  Pi  -  /i,-‘P(As  +  Dh(x)) . 

Again,  to  examine  the  system  (2.1)  with  the  control  strategy  (5.3)  and  the  approxi¬ 
mate  differentiator  (S.fi),  we  shall  refer  to  the  theory  of  decoupling  motions  ([21],  [24]). 

If  we  denote 

s  -  As  -i-  Dh(x)  ■«  Aiji  ,  (5.10) 

then  the  method  of  decoupling  motions  described  in  [21]  is  suitable  for  the  resulting  sys¬ 
tem.  We  now  examine  the  condition  for  the  convergence  of  the  fast  motions  to  the 
manifold  s  »  0. 

The  projection  of  the  overall  system  on  the  manifold  s  is  given  by 
s  <■  Ai  +  Dy 
»»  Ai  -I-  DHx 

+DH|f -l-G(u  -KOI  • 

Replacing  u  by  its  value  yields 

3  =  (A  -  DGHKP)m,~‘s  DH(f  -K  GKF  -K  GKu  -r  GO  •  (5.11) 

If  we  now  multiply  both  sides  of  the  above  equation  by  fx,,  and  let  t  ••  ix,t,  we  get 
-  (A  -  DHGKP)s  -K  M,DH(f  GKF  -K  G?  -K  GKi^)  , 

QT  Qt 

where  x  =  constant,  t  =*  constant.  If  the  matrix  [A  —  DHGKP]  is  Hurwitz  then 
lim  s  -  -  ^(A  -  DHGKPl'MDHf  -K  DHGK(F  +  i/i  +  DHG?]  . 

f^OO 

Using  twice  the  matrix  inversion  lemma  and  the  condition  —  PA~‘D  ■■  Im  we  obtain 
lim  (A  -  DHGKP  )■' 
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-  lim  p-*P(A-D(eHG)K»Pr* 

c-^oo 

-  lim  p-i(I  -  (K®  +  (cHGr‘r‘K®lPA-‘ 

C-^CO 

-  +  P“‘(HGKr*PA-*  . 

Hence 

lim  s  —  iU,P“‘(F  +  L)  ,  (5.12) 

f-^OO 

or  Air'Ps  “  F  +  i/,  which  gives 

y  -u^T{y,u)  . 

To  derive  the  equation  of  the  slow  motions,  we  let  /i,  equal  to  zero.  Hence  using 
equation  (5.9)  we  get 

z  -  —  A"‘Dh(x) , 
y  -  Pi  -_PA-‘Dh(x)  . 

Using  the  fact  that  —  P,A.“*D  <■  I,  we  obtain 

y  =  y  . 

Hence  the  equation  of  the  slow  motions  is  given  by 

X  -  f  +  G(K(F(y,t')  -  y  +  i')  +  f! 

=  f  +  G[u  +  ?1  , 

where 


u  -  K(F(y,u)  -  y  +  ^)  -  K(F(y,i/)  -  y  +  u)  . 


Remark  5.1:  Note  that  for  large  but  Enite  values  of  the  K-matrix,  the  value  of  the 
control  signal  u  remains  Enite  (as  shown  in  Proposition  5.1  part  (b)). 

INFLUENCE  OF  NOISE 

We  now  investigate  the  inEuence  of  noise  on  the  behavior  of  the  system  (2.1)  with 
the  control  law  (5.3).  Assume  that  the  output  vector  y  is  corrupted  by  the  continuously 
differentiable  noise  r(t],  thus 

y-y+r(t),  (5.13) 

We  now  find  values  for  y,  y ,  and  u.  We  assume  that  det(I  +  HGK)  #  0. 


(a)  Recall 


y  -  H(f  +  G(K{F(y,t/)  -  y  +  ^t))  +  f]]  , 
usinC  equation  (5.13)  we  get 

y-H(f  +  G{K(F-y-f +  £/)  +  ?]]. 

Sol-viag  for  y  we  obtain 

y  -  (I  +  HGK)-*  [Hf  +  HGKF  -  HGKf  +  HGKm  +  HG^]  .  (5.14) 

(b)  For  the  controller  u, 

u  «■  K(F  —  y  +  i/) 

-K(F  -  y-r  +  ^), 

substituting  y  —  Hf  +  HGu  +  HG^,  we  get 

u  -  K(F  -  Hf  -  HGu  -  HGif  -  r  +  , 

solving  for  u  yields 

u  «  (I  +  KHG)-*K(F  -  Hf  -  f  -  HG^  +  u) .  (5.15) 

(c)  The  derivative  of  the  output  vector  with  noise  is 

y  -  Hf  +  HGu  +  HG^  . 
using  u  =»  K(F  —  y  +  t')  we  obtain 

y  -  Hf  +  HGK(F  -  y  -u  !>)  +  HG^  , 

solving  for  y  yields 

y  -  (I  +  HGK)-‘(Hf  +  HGKF  +  HG^  +  HGKS')  .  (5.16) 

In  the  limit  the  equations  (5.14),  (5.13),  and  (5.16)  become 

(i)  y  -  F  -  f  +  t', 

(ii)  u  -  (HG)-‘  (F  -  Hf  -  r  -  HG^  +  i/j. 

(Hi)  y -t'-F(y,i/) . 

In  part  (i)  above  we  can  see  that  for  an  actual  system,  in  the  limiting  case,  the  noise  r(t) 
is  fully  "repeated"  in  the  output.  As  for  the  controller  u,  apart  from  the  "basic"  control 
law  u  —  (HG)~‘(F  —  Hf  —  HG^  +  i/),  we  have  an  additional  component  due  to  the 


additive  aoise. 


S.  CONCLUDING  REMARKS 

In  this  paper,  we  discussed  the  robustness  of  high-gain  output  feedback  control 
designs  for  nonlinear  time-varying  uncertain  models  to  unmodeled  high-frequency 
dynamics.  Our  approach  followed  on  the  papers  by  Vostrikov  et  al.  [24],  and  Utkin  [2j. 

Two  different  control  strategies  were  analysed.  The  first  one  was  concerned  with 
the  output  regulation.  To  facilitate  the  synthesis  we  utilised  a  diffeomorphic  state  vari¬ 
able  transformation  of  the  given  model  into  the  regular  form.  This  regular  form  was 
fotmd  very  useful  in  the  design.  However  the  problem  of  constructing  a  transformation 
which  brings  the  system  into  this  form  requires  further  investigation. 

The  aim  of  the  second  output  feedback  control  design  was  to  ensure  the  tr:  V:icg 
by  the  output  of  a  given  reference  signal.  The  proposed  control  algorithm  involved  the 
output  vector  derivative.  Following  Vostrikov  et  al.  [241,  we  suggested  a  sensor  estimat¬ 
ing  the  output  derivative.  One  may  argue  that  using  differentiating  filters  is  impracti¬ 
cal.  However  one  has  to  recognize  that  the  essential  information  about  a  given  process 
has  significant  spectral  components  only  at  low  frequencies  [13  p.  227],  Hence  if  we  use 
an  approximate  differentiator  which  is  sufficiently  fast  then  the  system  will  hardly  feel 
the  difference  between  the  ideal  and  approximate  differentiators.  Thus,  this  approxi¬ 
mate  differentiator  acts  as  an  ideal  one  and  its  gain  leveb  off  or  decreases  at  higher  fre¬ 
quencies.  In  this  paper  we  attempted  to  prove  that  the  approximate  differentiator  is  a 
viable  tool  in  the  synthesis  of  control  algorithms. 
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CONTROL  OF  UNCERTAIN  DYNAMICAL  SYSTEMS  : 
SIMULTANEOUS  STABILIZATION  PROBLEMS 

BiJoy  K.  GHOSH 
Washington  University 
Saint-Louis.  Missouri  63130,  U.S.A. 


In  the  last  decade,  significant  progress  have  been  witnessed  In  the 
design  of  a  robust  compensator  for  a  family  of  multi  Input  multi  output 
systems.  The  main  objective  Is  to  construct  a  dynamic  compensator  which 
simultaneously  stabilizes  a  family  of  plants  and  satisfies  various  other 
design  restrictions.  The  motivation  Is  to  extend  various  classically  well- 
known  compensator  design  methods  for  a  single  plant  to  a  family  of  plants . 
Such  a  family  of  plants  may  occur  as  a  result  of  parameter  uncertainty  or 
parameter  variation  In  Che  plants  and  the  goal  Is  to  construct  a  compensator 
which  Is  Insensitive  Co  Chase  parametric  changes. 

To  begin  with,  we  consider  the  “simultaneous  stabilization  problem” 
described  as  follows: 

Given  a  r  tuple  G^ . of  pxm  proper  transfer  functions,  does  there 

exist  a  compensator  i.(t)  such  that  the  closed  loop  systems  Cj^(I  -f 

KGj^]'^,...,  C^H  +  KG^)'^  ace  Internally  stable? 


This  problem  arises  In  reliable  system  design  where  Gj,..,,  G^  represent  a 

plant  Gj^  operating  In  various  modes  of  failure  and  K(s)  Is  a  non- switching 

stabilizing  compensator.  It  also  arises  In  the  stability  analysis  and 
design  of  a  plant  which  can  be  switched  into  various  operating  modes.  It 
has  been  shown  In  [1]  that 


The  Integer  max(m,p)  Is  the  critical  number  of  plants  below  which  the 
simultaneous  stabilization  problem  is  solvable  almost  always  l.e. 
genetically  (In  a  suitable  topology)  by  a  compensator  of  HcHlllan  degree 

where  la  the  asMllest  Integer  satisfying 


qQ(max(m,p)  ♦  l-r]  £ 


^  n.  -maxCm.p) 
l-l  *■ 


(1) 
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In  the  above  formula,  n^^  Is  the  McMillan  degree  of  the  plant  for 

1-1,..., r  respectively.  In  fact.  If  mln(m,p)  -  1  than  the  formula  (1)  also 
computes  the  minimum  order  of  the  generically  stabilizing  compensator.  It 
may  be  remarked  that  the  minimum  order  compensator  problem  Is  a  classically 
unsolved  problem  and  In  (1]  the  problem  Is  solved  for  the  special  case 
mln(m,p>  -  1. 

However,  beyond  saying  that  the  simultaneous  stabilization  problem  is 
solvable  for  certain  classes,  It  Is  of  great  Interest  to  parameterize  all 
chose  cases  where  the  problem  Is  Indeed  solvable.  Moreover,  for  ease  of 
computation,  such  a  parameterization  has  Co  be  explicit.  This  question  is 

parameterizing  the  sec  of  r  tuples  of  plants  (G^,...,G^)  Is  addressed  In 

[2]  and  one  of  his  main  results  Is  a  considerable  conceptual  breakthrough, 
since  to  check  simultaneous  scabllizablllcy  using  this  result  one  only  needs 
CO  know  which  path  eeaponenC  (G^ .  G^)  lies  in;  i.e.  the  problem  is 

reduced  to  the  problem  of  analyzing  big  pieces  of  Che  space  of  r  tuples  of 
systems  rather  Chan  Individual  r-Cuples.  Similar  results  on  simultaneous 
stabilization  and  pole  assignment  for  a  parameterized  family  of  plants  by  a 
parameterized  family  of  compensators  is  also  obtained  by  Dr.  Ghosh  and  is 
reported  in  (2).  To  my  knowledge,  use  of  semialgebraic  geometric  methods 
for  Che  purpose  of  parameterizing  scabilizable  or  unscabllizable  path 
components  has  been  done  for  Che  first  time  In  [2]. 

Considering  more  chan  max(m,p}  plants  for  the  purpose  of  simultaneous 
stabilization  (is  quite  a  reasonable  objective  in  robust  system  design),  but 
unfortunately  in  particular  in  (3]  it  is  shown  chat,  "Fairs  of 
simultaneously  stablllzable  single  Input  single  output  plants  of  bounded 
McMillan  degree  may  not  have  simultaneously  stabilizing  cos^ensacors  of 
aprlorl  bounded  McMillan  degree . * 

It  Is  shown  by  Dr.  Ghosh  In  [3]  that  there  exists  a  sequence  of  pairs  of 
simultaneously  stablllzable  plants  of  degree  one  for  which  Che  minimum 
degree  of  the  stabilizing  compensator  Is  arbitrarily  large.  A  consequence 
of  Che  above  proposition  Is  chat  a  simultaneously  stabilizing  compensator 
cannot  be  constructed  by  solving  a  set  of  simultaneous  equatloi.s  or 
inequalities  In  the  coefficients  of  a  parameterized  family  of  compensators 
I  of  a  given  McMillan  degree.  Stated  differently,  if  r  >  max(m,p),  the 


I 


/ 
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classtcslly  known  algebraic  and  aealalgabcalc  gaoaatxie  aethoda  are 
Inapplicable  since  Che  coapensacor  apace  ta  not  £in5.ce  dlaanaienal  and  in 
particular,  any  nuaerlcal  coapucaCion  of  the  aaaoclaced  coapensacor  needs  co 
use  a  aora  appropriate  cranscandancal  aechod  proposed  by  Dr.  Ghosh  in  [4], 
Also  In  [4]  a  new  'partial  pole  placaasnC  problsa  is  proposed  which  arises 
froa  a  aore  practical  design  requireMnc  to  place  an  arbitrary  nu^er  of 
self  conjugate  poles  in  Che  closed  loop  while  restricting  the  reaalning 
poles  in  the  region  of  stability.  The  following  result  is  shown: 

The  problem  of  slaulcaneously  stabilizing  three  single  input  single 
output  plants  chosen  genacically  is  equivalent  to  the  problea  of  partially 
pole  placing  one  single  input  single  output  plant  by  a  stable  ainiaua  phase 
compensator. 

Use  and  application  of  a  stable,  miniaua  phase  compensator  is  introduced  in 
[4]  for  Che  first  time.  Furthermorea  folklore  example 

,  a,: 7..,  .  a- 2.  .  _ liS — 

s-4.6  2S-2.6  4.8S-24.6 

of  a  triplet  of  simultaneously  unscabilizable  plants  that  are  scablllzable 
in  pairs  is  constructed  by  Dr.  Ghosh  [4].  These  results  to  multi  input 
multi  output  problems  are  further  generalized  in  [4]  co  show  Chat 

"If  r  ain<m,p)  s  m+p,  the  simultaneous  partial  pole  assignment  problem 
may  be  analyzed  via  interpolation  methods  and  one  obtains  a  semialgebcaic 
parameterization  of  the  partially  pole  assignable  r-tuples  of  plants.  If  r 
aln(m,p)  >  m-fp,  the  simultaneous  partial  pole  assignment  problea  is  to  be 
analyzed  via  transcendental  methods  introduced  in  [4]." 

The  above  result,  therefore,  characterizes  the  "degree  of  difficulty*  end  In 
particular  assarts  the  existence  of  carcaln  cases  (say  for  example  a-p,  rml) 
when  interpolation  methods  are  inapplicable  in  the  siaulcanaous 
stabilization  problea. 

We  have  seen  so  far  chat  transcendental  aethods  are  useful  when  the 
degree  of  the  coapensators  under  consldaratlon  is  not  aprlori  bounded. 
Frequently  in  syscaa  Identification  and  control,  it  is  of  interest  co  study 
a  family  of  plants  for  which  Che  McMillan  degree  is  not  fixed.  In 
particular  the  degree  aay  degenerate  to  a  lower  value.  Thus  rather  chan 
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fixing  che  McMillan  degraa  of  a  plant,  it  la  uaaful  Co  pacaaeCeclze  plants 
of  McMillan  dagraa  £  n  foe  aona  n.  Wo  .chacafoca,  poaa  cha  folloving 
quaaclon  •• 

"Paraaatarizo  cha  sot  0^  of  plants  of  dagraa  s  n  (possibly  aa  a 
aanlalgabraie  anbaoc  of  an  algabcaic  aac)  such  that  a^ocy  p  in  hat  an 
opan  neighborhood  N<p)  of  p  in  0^  such  Chat  M(p>  is  alBultanaoutly 
atabillzabla  by  a  conpanaator  of  dagraa  s  q  for  aona  q." 

Noca  chat  this  quaatlon  posas  robust  stabilization  as  a  paraaeterlzaclon 
problan.  In  (S)  an  axpllclc  parameterization  of  0^  is  obtained  as  a 

subset  of  for  the  single  input  single  output  systems  and  in 

particular  we  show  that  •• 

"Assume  n-p>l,  than  «n  a  aamlalgobrale,  opan,  connected  and  dense 
subset  of 

More  surprisingly  we  show  chat 

"(1  is  a  trivial  vector  bundle  over  a  circle.  In  particular  (1  is 
dlffeomorphic  to  x  IR^".- 

The  space  has  been  parameterized  for  a  multi  input  multi  output 

plant  in  [6]  as  a  vector  bundle  over  a  Grasananian,  a  well  known  object  in 
algebraic  geometry.  We  argue  that  R^  and  not  rat  n  (cha  apace  of 

strictly  proper  single  input  single  output  cransfer  functions  of  a  given 
n 

degree)  or  ^  (che  space  of  pxa  transfer  functions  of  degree  n)  is  a 

m.p 

more  natural  space  for  system  identification  and  control.  Various 
properties  of  this  space  has  been  reported  in  (8]. 

The  geometry  of  R^  is  useful  in  che  study  of  a  structured  family  of 

plants  wherein  che  degree  is  apriori  bounded.  In  practice,  however,  one  is 


also  incarastad  In  tha  study  of  a  family  of  plants  possibly  with  soma 
unmodellad  dynamics.  For  axampla.  undar  tha  prasanea  of  a  high  fraquency 
'parasitica*  it  is  unraasonabla  to  aaauma  that  tha  McMillan  degraa  of  a 
faodly  of  plants  la  bounded  by  n.  In  (6)  wa,  tharafora,  construct  the 
space  as  a  direct  limit  of  tha  spaces  C  O2  C  ...  whare  0^  is  a 

subspace  of  IR**.  Of  course  two  points  in  0^  can  nodal  the  same  dynamical 

system  and  one  therefore  considers  the  quotient  space  where  two  points 
in  0^  are  equivalent  if  they  correspond  to  tha  same  dynamical  system. 

Various  properties  of  are  being  studied.  In  particular,  we  show  chat 

in  S  there  exists  arbitrary  small  open  neighborhood  H  with  the 
following  property- - 

There  exists  a  sequence  plants  in  M  such  that  tha 

minimum  degree  of  the  stabilising  dynamic  compensator  for  the  plants 
eorcesponding  to  fg,  increases  arbitrarily. 

This  fact  In  particular  Implies  that 

"There  exists  p  €  llT  such  that  every  open  neighborhood  N  of  p  in 
cannot  be  stabilized  even  by  an  adaptive  controller  of  arbitrary  large 
degree  q.* 

Thus  we  obtain  a  major  limitation  of  the  adaptive  controllers  that  are 
currently  of  interest  In  system  theory,  viz.  open  neighborhoods  of  points  In 

that  cannot  bo  robustly  stablllzable  even  by  an  adaptive  controller. 

The  structure  of  also  enables  us  to  define  a  hybrid  family  of  plants, 

(l.a.  a  family  of  planta  with  structured  and  unstructured  uncertainty).  In 
particular  in  [6]  we  characterize  (for  the  first  time  in  the  literature) 
hybrid  families  of  plants  that  can  be  stabilized  sisnilcanaously  by  an 
adaptive  controller. 
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Th«  propo*ed  hybrid  paranatarizatlon  ha*  many  advantagaa  ovar  tha 
eurrantly  axlstlng  graph  paranacarlzatlon  dua  to  Vldyatagar.  In  fact  tha 
hybrid  paranatarlzatlon  Is  gradad  by  tha  dagraa  o£  tha  dynaalcal  syatans  and 
aaeh  ona  of  tha  gradad  space  Is  dlffaonorphlc  to  an  Euclidean  space  If  tha 
plant  Is  strictly  proper.  The  Euclidean  structure  Is  of  particular 
importance  In  ayscan  Identification.  Furtharaora,  tha  sequence  of  plants 
for  axaaple 


^n+1 


converges  to  ^  as  n-«  in  the  graph -topology.  Thus  In  graph 

parameterization,  arbitrary  close  to  a  plant  of  a  given  degree  there  exists 
plants  of  arbitrary  large  degree  which  Is  clearly  a  deficiency  from  the 
point  of  view  of  robustness  and  obtaining  an  aprlorl  bound  on  the  complexity 
of  the  compensators.  Hybrid  parameterization  does  not  suffer  from  chase 
disadvantages  and  therefore  appears  to  be  a  good  parameterization  for  system 
Identification  and  adaptive  control. 

In  {7]  we  study  the  problem  of  simultaneous  stabilization  of  a  family 
F  of  plants  described  as  follows  •• 


F  S  (g(s):  g(s)  -  t  I  a  s^]  /  (  £  b.s^ 

i_0  ^  1-0 

7^  s  i-0 .  n-1,  deg  g(s)  -  n| 


+  »"]. 


Ve  prove  the  following  rather  surprising  result 


"A  necessary  and  sufficient  condition  that  every  plant  In  F  Is 
simltaneously  stablllzable  by  a  feedback  gain  k  Is  that  eight  planes  In 
F  (suitably  chosen)  Is  simultaneously  stablllzable  by  a  feedback  gain  fc." 


We  find  Che  above  result  quits  surprising.  Indeed  It  asserts  the  existence 
of  a  suitable  family  of  uncountably  many  plants,  stablllzsblllty  of  which 
can  be  asserted  via  the  simultaneous  stabilisation  problem  of  a  finite 
number  of  plants.  This  ve  consider  Is  s  major  conceptual  breakthrough. 

The  main  Idea  of  the  preceding  paragraph  can  be  generalized  to  Include 
dynamic  coispansaclon  as  well.  In  fact  one  can  obtain  a  sufficient  condition 


89 


which  can  be  made  aayaptoclcally  nacaaaary  by  Incraaalng  the  eonpucatlonal 
conplexlty  of  Che  algorithm.  Roughly  speaking  one  therefore  concludes  the 
existence  of  a  computational  technique  to  eonsccucc  a  robust  compensator 
which  can  be  asymptotically  Improved  by  considering  Increased  eomputaClonaX 
load.  This  In  my  view  Is  a  computational  breakthrough  and  In  particular 
such  a  sequence  of  algorithms  did  not  exist  in  the  literature  previously. 

For  the  purpose  of  constructing  a  compensator  with  an  aprlorl  bounded 
McMillan  dagrea  It  Is  Important  Co  consider  to  following  problem. 

"Given  a  family  F  of  linear  dynamical  syatams  Chat  can  ba  stablllxrij 
slmultanaottsly  by  a  fixed  non-swltchlng  compensator.  Does  there  exist  an 
aprlorl  bound  on  the  degree  of  the  compensator  irtileh  simultaneously 
stabilises  F." 

In  general  Che  above  problem  la  unsolved.  However  for  a  1  parameter 
family  of  plant  we  have  a  surprising  result;  Let  x^(s)/y^(s)  and 

X2(s)/y2(s}  be  a  pair  of  proper  but  not  scrietly  proper  plants.  Consider  a 
1  parameter  family  F  of  plants  described  as  fellows 

F  -  g;^<a)  -  (A*!  +  (l*A)X2)/[*yi  ♦  (l**)y2l 

A  e  (0,  1).  deg  g;^(s)  S  n  V  A). 

Let  . a^  denote  Che  zeros  of  '*■■^2  *  *2^1  ****  open  left  half  of 

Che  complex  plane.  Lee 

bj  -  X2/X2(aj^)  If  Che  mulclpllclcy  of  a^  as  a  common  zero  of  x^, 

Is  S  mulclpllclcy  of  a^^  as  a  common  zero  of  y^,  y2 
“  otherwise. 

for  1-1,  ...,  t.  Let  Sj,  -  (aj-l)/(aj+l)  and  where 

Che  branch  cue  for  cha  square  rooc  Is  taken  to  be  Che  non-poslclve  real 
axis.  Fucchemore  let  k  be  the  largest  real  number  such  chat 

(1  -  k\zj]/ll  -  SjSjjJ^ 

la  non-negative  definite.  The  sMiln  result  Is  new  described  as  follows 
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"The  following  three  stetenenta  ere  equivalent. 

1.  F  ia  aianiltaneoualy  atabiliaable  by  aome  dynamic  compenaator. 

2.  F  ia  aimultaneooaly  atabiliaable  by  aome  dynamic  compenaator  of 
degree  S  3n-2. 

3.  k  >  1 

We  find  that  the  above  reault  ia  quite  aurprising.  In  fact,  where  aa 
the  conjecture  •  "pairs  of  almultaneoulsy  stabillzable  plants  of  bounded 
HcHlllan  degree  have  simultaneously  stabilizing  compensators  of  bounded 
McMillan  degree*  -  is  falsa,  Che  conjecture  Chat  "aimulcaneously 
stabillzable  linear  1-paramecer  family  of  plants  of  bounded  McMillan  degree 
have  simultaneously  stabilizing  compensators  of  bounded  McMillan  degree"  is 
Indeed  true.  Of  course  it  is  unknown  if  similar  results  would  continue  to 
be  true  for  multiparameter  family  of  plants.  It  appears  however,  in  view  of 
the  above  result,  that  Che  problem  of  stabilizing  a  discrete  r- tuple  of 
plants  (in  particular  a  pair  of  plants)  aiiaultaneously  is  a  inich  harder 
problem  to  solve  compared  Co  simultaneously  stabilizing  a  continuous  family 
of  plants.  This  fact  Indeed  appears  to  be  quite  contrary  Co  our  original 
expectation  •  in  fact  Che  problem  of  simultaneous  stabilization  of  a  pair  of 
plants  was  originally  used  with  an  idea  of  simplifying  the  robust 
stabilization  problem  of  a  family  of  plants. 

In  order  to  arbitrary  tune  the  closed  loop  frequencies  of  a  plant,  it 
is  necessary  to  consider  the  simultaneous  pole  assignment  problem.  In  [6] 
we  analyze  the  pole  placement  problem  as  an  intersection  problem  and  apply 
Schubert  enumeracive  calculus  to  compute  (under  appropriate  cases)  the 
number  of  complex  dynamic  compensators  that  would  place  Che  closed  loop 
poles  of  a  set  of  r-plancs  in  a  given  sac  of  self-conjugate  complex  numbers. 
We  compacclfy  the  space  of  cootpensacors  and  define  a  set  of  points  known  as 
'base  locus'  and  a  set  of  points  known  as  'critical  points.'  Roughly 
speaking,  we  assert  in  [6]  chat  a  compensator  has  to  avoid  Che  base  locus 
and  Che  critical  points  for  otherwise  Che  closed  loop  response  of  the 
control  system  would  either  be  sensitive  or  would  fall  to  bo  robust  with 
respect  to  changes  in  the  parameters.  An  explicit  parameterizaclon  of  chase 
points  also  open  up  some  new  restrictions  in  the  compensator  design  problem 
previously  unknown  in  system  theory. 
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To  sviamarlze,  we  aaintaln  thac  Che  uaa  of  seBlalgebralc  gaoaetric, 
algebraic  geonacric  and  transcandancal  aathods  are  three  dlatlnct 
foundational  technlquea  that  have  been  applied  In  robuat  ayataa  dealgn. 
Extensions  of  these  aathods  to  paraaetarlzatlon,  design,  Identification 
probleas,  and  adaptive  control  %rauld  be  useful  and  are  currently  being 
explored.  These  techniques  are  also  being  extended  to  nonlinear  and  tlae 
varying  systaas. 
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ROBUST  MODEL  TRACKING  FOR  A  CLASS  OF  SINGULARLY  PERTURBED 
NONLINEAR  SYSTEMS  VIA  COMPOSITE  CONTROL 

P.  Garofalo  and  L.  Gllelmo 

DiparclmenCo  dl  Infoewatica  e  SistemLstlca 
Universica’  degll  Studi  di  Napoli 


1,  Introduction 


Typical  problems  encountered  in  the  design  cf  a  control  system  are  the 
presence  of  parameter  uncertainties  and  the  coexistence  of  slo«  and 
fast  dynamics  in  the  plant  to  be  controlled.  When  the  uncertainties  are 
described  assigning  their  range  of  variation  and  these  variations 
belongs  to  appropriate  subspaces,  the  so  called  deterministic  control 
of  uncertain  systems  (Leitmann,  1980;  Corless-Leitmann,  1981) 
represents  an  useful  tool  for  the  design  of  controllers  capable  of 
guaranteeing  certain  performance  no  matter  what  the  realization  of  the 
uncertainties  is.  The  rigorous  treatment  of  systems  with  two-time  scale 
behavior  can  be  done  utilizing  singular  perturbation  theory  (Kokotovic 
et  al.;  1986).  The  simultaneous  use  of  these  two  methods  for  the 
control  of  uncertain  two-time  scale  systems  has  recently  received  some 
attention  (see  Leitmann  (this  volume)  and  its  references). 

In  this  paper  we  use  a  composite  control  technique  in  conjunction  with 
the  robust  design  of  controllers  for  uncertain  systems  to  synthesize  a 
nonlinear  controller  which  forces  a  class  of  two-time  scale  nonlinear 
system  to  follow  a  two-time  scale  linear  reference  model.  The 
controllers  that  are  used  in  the  two  phases  of  the  design  are  obtained 
via  a  constructive  use  of  Lyapunov  functions  (Kalman-Bertram,  1960). 
The  same  Lyapunov  functions  are  successively  combined  (as  suggested  by 
Saberi-Khalil ,  1984)  for  obtaining  the  proof  of  ultimate  boundedness  of 
the  model  tracking  error. 
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2.  Problem  Statement 


We  consider  a  two-time  scale  nonlinear  system  described  by  the 
following  equations 

x(c)  -  A^^(x(t))x(c)+A^^(x(C))z(t)+B^(x(t))u(c)-^a^Cx(t))  ; 

(2.1a) 

MZ(t)  -  A^^(x(t))x(t)+A^^(x(t))z(C)-t-B^(x(t))u(C)-Ka^(x(c))  ; 

(2.1b) 

x(t^)  -  x^;  (2.1c) 

z(t^)-z^;  (2. Id) 

where  x(t)€R''  ,  z(c)eR“  represent  the  state  of  the  system,  u(t)6R’’  is 
the  control  input,  a^(x(c))  and  aj(x(t))  are  nonlinear  vectors,  m6(0,“) 
is  Che  singular  perturbation  parameter,  and  ( • )  and  B^(  ).  i-1,2, 
J-1,2  are  matrices  of  appropriate  dimensions. 

The  reference  model  specifying  the  state  behavior  expected  from  the 
controlled  plant  is  described  by  the  linear,  time -invariant  system 


x(c)  -  &^^x(t)+S,^z(t)+6^u(C) ; 
pz(t)  -  &^^x(t)+S^^z(c)+6^u(c) ; 


(2.2a) 
(2.2b) 
(2.2c) 
(2. 2d) 


where  x(t)6R'’  and  z(c)eR”  is  the  state  and  u(t)eR'’  is  a  reference 
signal . 


The  following  assumptions  define  Che  class  of  nonlinear  plants 
considered  here. 


Assumption  1.  There  exist  full  rank  matrices  B  ,  i-1,2  such  chat,  for 

I 

all  xeR  ,  Che  following  decomposition  holds: 


J. 
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B^(x)  -  +  B^E^(x)  ,  i-1,2, 

a  (X)  -  B  d  (X)  ,  1-1,2  , 

ill 

where  (resp.  d^('))  Is  a  matrix  (reap,  a  vector)  of  appropriate 

dimensions,  continuously  differentiable  with  respect  to  x. 


The  relationship  between  the  system  (2.1)  and  the  reference  model 
represented  by  equations  (2.2)  is  precised  by  the  following 
assumptions. 

Assumption  2.  For  all  xeR"  the  following  equalities  hold 


A  (x)-&  -  B  C  (x)  , 

tJ  ij  i  ij  ■ 


i,J  -  1.2 


B  -  B  C 

I  1  1 


i  -  1,2 


where  C  ^(x)  are  continuously  differentiable  matrices. 


Moreover,  the  singularly  perturbed  model  is  assumed  in  scandard  form, 
i.e.  , 


Assumption  3.  Matrix  is  full  rank. 


Defining 


0  11 


.  A 

12  22  21 


(2.3) 


we  hypothesize  that 
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Assumption  A.  The  pairs  and  are  controllable. 

Assumption  5.  The  matrices  A^^(x),  B^(x),  a^(x),  for  1-1,2  and  J-1,2, 
are  norm  bounded  in  R"'.  In  particular  we  define 

-  s“P„l|C  (x)i. 

<c,  -  sup  llE  (x)|, 

"  xSR"  ‘ 

xeR 

Moreover  ic  <  1  ,  1-1 , 2  . 

i 

Finally  we  make  the  following 

Assumption  6.  The  Input  reference  signals  u( ' )  are  such  that  there 
exist  finite  positive  constants 


k  -  sup  llu  (c)||, 


k  -  sup  llu  (Oil. 
te(c^,«)  ' 


where  u^(t)  and  u^(t)  represent  the  slow  and  the  fast  time  scale 
components  of  u(t)  and  u(c)^  (t)+u^(t) .  Corresponding  to  these 
signals,  there  exists  a  positive  constant  ft  such  that,  for  the 
state  variables  of  the  reference  model  are  uniformly  bounded  by  known 
constants : 


kA  -  sup  llx(t)ll, 

t6[t^ ,•) 
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kA  -  sup  ||2(t)||. 

Remark  1 .  Assumption  1  Is  the  so  called  "matching  assumption”  and 
defines  the  manner  in  which  the  nonlinearities  enter  the  plant.  The 
equalities  in  Assumption  1  and  2  are  the  so  called  "model  matching 
conditions"  and  determine  the  class  of  model  that  can  be  tracked  by  the 
nonlinear  system  under  consideration. 

Remark  2.  System  (2.1)  belongs  to  the  class  of  singularly  perturbed 
nonlinear  system  with  slow  nonlinearities  considered  by  Chow-Kokotovic 
(1981).  Note,  however,  that  for  design  purposes,  it  is  not  strictly 
necessary  to  know  the  nonlinearities  affecting  the  system  but  only  a 
nominal  linear  behavior  and  an  evaluation  of  the  maximum  deviation  from 
this  behavior  as  precised  in  Assumption  5.  The  composite  control  design 
for  the  practical  stabilization  of  a  similar  class  of  plants  is  also 
considered  by  Garofalo  (to  appear). 

The  objective  of  the  control  is  to  synthesize  a  feedback  control 
function  guaranteeing  that  the  plant  tracks  the  model  to  within  a 
bounded  neighbourhood  of  the  zero  state  tracking  error. 

The  procedure  we  propose  for  the  synthesis  of  the  controller  is  based 
on  the  separate  design  of  controllers  guaranteeing  tracking  of  the 
slow  approximation  and  of  the  the  boundary  layer  approximation  of  the 
reference  model.  On  the  basis  of  these  control  laws  the  composite 
control  is  constructed  which  guarantees  cracking  of  the  model  for 
sufficiently  small  values  of  the  singular  perturbation  parameter  n. 


1 


A  formal  definlelon  can  be  found  In  Corless  (1987)  or  in  Appendix  1. 
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3.  Slow  Time  Scale  Control 


Following  KokoCovic  et  al.  (1986)  the  slow  approximation  of  the 
behavior  of  the  reference  model  Is  obtained  considering  n-O  In  (2.2b} 
and  substituting  the  resulting  value  for  variable  z  In  (2.2a), 
obtaining 

x(t)-&x(c)+Su(t),  (3.1) 

«  Os  0  t 


where 


0 


-  i  , 
12  22  2 


(3.2) 


and  the  subscript  s  stands  for  slow  cinie>scale  approximation. 

In  order  to  design  the  controller  for  tracking  the  slow  component 
(3.1)  of  Che  reference  model,  we  need  an  approximation  of  system  (2.1) 
in  Che  slow  time  scale.  To  this  end,  we  assume  Chat  z  variable  has  a 
nominal  behavior  z  which  is  exactly  the  one  that  z  variable  cakes  in 

n 

the  reference  model,  chat  is 


pz  (t)  -  &  x(t)+^  z  (c)+Su(c).  (3.3) 

n  21  22  n  2 

Correspondingly,  Che  approximate  model  of  slow  dynamics  neglects  the 

2 

nominal  fast  transients,  i.e.. 


X  -  A  (x  )x +A  (X  )z +B  (X  )u +a  (X  ) , 


0  -  &  X  z  u  , 

21  •  22  n  2  • 


(3.4a) 

(3.4b) 


2 

Soaecimas,  when  no  confusion  is  likely  co  occur,  we  delete  Che  time 
argument  of  the  functions. 


'5?' 
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which  gives 


z  -  r(x 


)  -  -A  (Ax 
1  22  21  * 


.  A  . 
) 


(3.5«) 


and 


X  -  A  (X  )x  +B  (X  )u  -A  (X  +a  (X  ) , 


with 


A  (x  )  -  A  (x  )  -  A  (X 

0  •  11  s  12  t  22  21 


Define  now  the  slow  time  scale  Cracking  error  as 


-A  A 
f  -  X  -X 

a  a  a 


(3.5b) 


(3.6) 


(3.7) 


On  Che  basis  of  (3.1),  (3.2),  (3.5)  and  (3.6)  Che  slow  time  scale 

cracking  error  dynamics  can  be  wrlccen  as 


^a  -  V.  ^  8l“a  ^  "  ®lt»i(*a>^^’«a  ^ 


+  B  (H  (X  )x  -H  (X  )u  +d  (X  )), 

llcs2s»lt 


(3.8) 


where  F  -  &  -B  K  ,  K  sR**”  is  a  matrix  which  makes  matrix  F 

s  0  1  »  •  « 

asympcoclcally  stable  with  specified  eigenvalues  (which  is  always 
possible  by  virtue  of  Assumption  4),  and 


H^(x.)  5  1C,,(X.)-C^^(X,)&;;&^^1, 

(3.9a) 

H/x,)  S  [VS2(^^^;2^2>- 

(3.9b) 

From  the  knowledge  of  matrices 

Assumption  2) ,  and  matrices  k  ,  k  and 

22  21 

and  (given  in 

6^,  we  can  compute  the 

following  constants 

f 


S  sup  ||H(x)+k1|.  O.lOa)^ 

xea"  ' 

-  sup  IlH  (x)x  -H  (x)u+d  (x)i.  (3.10b) 

d  "  »  2  ml" 

1  x€B. 

Consider  now  the  nonlinear  feedback  control  law 

(Ambrosino-Celentano-Garofalo,  1985;  Garofalo-Glielmo ,  to  appear) 


p  (^  )  ^  -7  ?P  f  .  (3.11a) 

St  s  1.  «  s 

where  P  is  the  solution  of  the  Lyapunov  equation 

S 

F^P  +  P  F  -  -Q  ,  Q  positive  definite,  (3.11b) 

St  t  s  s  s 

and 


|BjP,«  Jl  +  «. 


(3.110 


This  feedback  control  has  the  tracking  capabilities  described  in  the 
next  theorem. 


Theorem  1.  Consider  the  slow  approximation  (3.4)  of  system  (2.1) 
subject  to  the  feedback  control  law  in  (3.11).  If  constants  7^^,  1-1,2, 
in  (3.11c)  are  chosen  so  as  to  satisfy 


(3.12a) 


3 


Notice  that  the  supreme  can  alvtys  be  raplacad  by  upper  bounds. 
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7 


1-ie 

1 


(3.12b) 


then  system  (3.5b)  tracks  the  slow  approximation  (3.1)  of  the  reference 
model  (2.2)  to  within  a  spherical  neighbourhood  of  whose  radius 

can  be  made  arbitrarily  small  by  a  suitable  selections  of  constants 
7  ,  i-1,2.  and/or  of  constant  S  in  (3. He). 

Si  S 

?roof .  The  proof  of  the  theorem  can  be  fo\md  In  Appendix  2. 


4.  Fast  Time  Scale  Control 


The  boundary  layer  approximation  of  the  reference  model  (2.2)  is  given 
by  (Kokotovic  ec  ai.,  1986) 


dz 

i  A  A 
■T—  —  A  Z  + 

df  22  t 


)  A 


(4.1) 


where  r-c//i,  represents  the  fast  component  of  the  reference  signal, 
and 


A 

z 

i 


A  A 

-  z 


_  ,  A  A  - 

r(x,u  ) 


A  A-l,A  A  A  A  . 
z  +  A  (A  x+B  u  )  . 
22'  21  2  « 


(4.2) 


The  fast  time  scale  approximation  of  system  (2.1)  is  obtained 
substituting  the  slow  control  expression  (3.11a)  in  equation  (2.1b)  and 
approximating  variable  x^(t)  by  x(t)  and  x^(t)  by  x(t).  So  doing  wo 
obtain 


(iz  -  Ajj(x)x-7^B^(x)rP_?  +  Ajj(x)z  +  B^(x)u^+a^(x) . 


(4.3) 


where  u^  is  the  fast  component  of  the  control  law  and  {  -  x-x. 
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Defining 


z 

t 


A 


z 


r(x.u  ) 

s 


2  +  &  *  (&  X+S  U  )  , 
22  21  2  •  ■ 


the  boundary  layer  model  of  the  system  can  be  vnrltten  as 


+B  G  <x)x+B  G  (x)u  +B  d  (x)  , 
2  1  2  2  •  2  2 

with 


Gj(x)  ^ 
Gj(x)  ^ 


[C  (X)  -  C  (x)&'‘^  ] 
*•  2l'  '  22'  '  22  21' 


-[6+C  (X)&‘^S]. 
■■I  22  22  2' 


(4.4) 


(4.5) 


(4.6a) 


(4.6b) 


The  fast  time  scale  tracking  error  can  be  defined  as 


(4.7) 


and,  on  the  basis  of  (4.1)  and  (4.5),  Its  dynamics  can  be  written  as 


-  - 

-  Ff  +  Bu  +  BE  (x)u  +  B  [G  (x)  +  K  )f  + 
dr  f’  £  2  £  2  2  £  2*  22'  £'  “  £ 

+  B^[G^(x)  -  7_(I^  +  Ej(x))rpj« 

+  B^[G^(X)X  +  c^(x)z^  +  Gj(x)u  -  +  dj(x)]  , 

(4.8) 

where  F  -  &  -B  K  and  K  eR*”  is  a  matrix  which  makes  matrix  F, 
asymptotically  stable  with  specified  eigenvalues  (see  Assumption  4). 


On  the  basis  of  Assumptions  1,  2,  5  and  6,  we  can  evaluate  the  finite 
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constants 

k  -  sup  |(G  (x)x  +  C  (x)2  +  G  (x)u  -  ft  u  +  d  (x)||, 

S  xell”  I  22  t  2  •  2  t  2  " 

te[t^,«) 

iteiO.ti) 

(4.9a) 

k  ft  sup  |C  (X)  +  K  II.  (4.9b) 

^  xeR”  “  ‘ 

k  ft  sup  ||G  (X)-  7  (I  +  E(x)}f  ?  I,  (4.9c) 

^  ‘  t  p  2  1  » 


In  the  fast  time  scale  the  variables  x  and  $  can  be  considered 
constants,  and  Che  fast  control  law  we  propose  for  making  the  boundary 
layer  system  crack  the  boundary  layer  reference  model  has  the  form 

Pj(f,)  -  (4.10a) 

where  is  the  solution  of  the  Lyapunov  equation 

F*P^  +  P^F^  -  -Q^,  positive  definite,  (4.10b) 


and 


Tj  -  7j(irjll 


+  «. 


s>o. 


(4.10c) 


Ue  can  state  the  following 

Theorem  2.  Consider  the  boundary  layer  approximation  (4.S)  of  system 
(2.1)  subject  CO  Che  feedback  control  law  (4.10).  If  constants  7^^, 
i-1 . 3  are  chosen  so  as  to  satisfy 


(4.1U) 


7 


tl 


£ 


7 


a 


£ 


7 


{3 


£ 


(4.11b) 


(4.11c) 


Chen  syscem  (4.5)  cracks  che  boundary  layer  reference  oodel  (4.1)  to 
vlchln  a  spherical  nelgbourhood  of  whose  radius  can  be  made 

arbicrarily  small  by  a  sulcable  selecclon  of  conscancs  7^^,  1-1 . 3 

and/or  conscanc  in  (4.10c). 

Proof.  The  proof  can  be  found  in  Appendix  2. 


5 .  Guaranteed  Performance  of  the  Composite  Control 


The  composite  control  is  obtained  as  the  sum  of  che  slow  and  the  fast 

A  A 

control  law  with  variable  replaced  by  f - [r(x,u)+r(x,u) ]  -  C  + 
^12^21^’  approximation  (,  obtaining 


u  -  -yj'pf  -  7.5*?  f  -  (, 

o  '•  1  •  't  2  't  2  «  22  21' 

where  f  »  (z-t) . 

For  this  control  law  we  can  establish  che  following  theorem. 


(5.1) 


Theorem  3.  Consider  system  (2.1)  subject  to  che  control  law  (5.1).  The 
closed  loop  system  cracks  che  reference  model  to  within  a  spherical 
neighbourhood  of  che  zero  state  cracking  error,  If  the  following 
conditions  are  satisfied. 

1)  The  constant  7^^  satisfies  che  Inequality: 


lOS 


with 

k.'  £  sup  ||H(x)x  -  H  (x)u  +  d(x)|(,  (5.3) 

1  xeR  *  ^ 

;»€(0,p) 

and  the  constant  7^^  satisfies  the  inequality  (3.12b); 

II)  the  constants  7^^,  j-1,2,3  in  the  control  law  (5.1)  are 

chosen  so  as  to  satisfy  Inequalities  (4.11); 

III)  constant  7^^,  besides  satisfying  (4.11c),  satisfies 

^  ^  (Q  ) 

<  2iP  irsupjB  (x)l|  (5.4) 

xeR 

Iv)  the  singular  perturbation  paraaeter  Is  such  that  (XitKti' 
where  m  is  a  constant  whose  value  can  be  a  priori  computed. 


Proof.  The  proof  of  Theorem  3  and  the  expression  for  the  upper  bound  of 
paramorer  p  are  given  In  Appendix  3. 


6.  Conclusions 


The  robust  model  tracking  control  presented  hers  Is  designed  using  the 
approach  of  datarmlnlstlc  control  of  uncertain  systems,  together  with 
the  composite  control  technique  developed  for  singularly  perturbed 
systems.  This  enables  the  designer  to  guarantee  the  modal  following 
within  a  spherical  neighbourhood  of  the  zero  error,  In  the  presence  of 
’slow*  nonllnearltles .  It  must  be  pointed  out  that  this  technique  does 


106 


not  require  the  knowledge  of  the  form  of  the  nonlineerlties ,  but  Just 
the  possible  range  of  their  variations. 


Appendix  1 


Some  definitions  and  a  useful  lenana. 

Consider  the  equation  of  a  model  tracking  error  dynamics  In  the  form 

e  -  v(t,t)  ,  «(C  )-<  ,  (Al.l) 

0  0 

where  tsS.,  <eR^,  and  Ve  say  that  the  system  tracks  the 

reference  model  to  within  a  spherical  neighbourhood  of  radius  R  of  <-0 
(Indicated  with  B(R))  Iff  the  following  properties  are  satisfied: 

1)  Existence  of  the  solution.  Given  any  (<  ,t  )€R'’xR  there  exists  a 
solution  <(•) ;  [t^,tj^)-*R'’,  t^>t^  of  (Al.l). 

11)  Indefinite  extension  of  solution.  Every  solution  «(•):[ ,tj^)-»R‘’ 
of  (Al.l)  has  an  extension  over  [t^,«). 

Ill)  Global  uniform  boundedness.  Given  any  bound  r&^,  there  exists  a 

bound  d(r)€R  such  that  If  <(■):[(  ,t, )->R^  is  a  solution  of  (Al.l)  with 
♦  OX 

llf^llsr,  then  ||«(t)||5d(r)  for  all  t6(c^,t^). 

Iv)  Local  boundedness  within  B(R) .  There  exists  a  spherical 

neighbourhood  B(R^)  of  «-0  such  that  If  <(  • ) :  [t^.  tj)-»R’’  is  a  solution 

of  (Al.l)  with  e  eB(R  )  then  <(t)eB(R)  for  all  tE[t  ,t  ). 

0  0  0  1 

v)  Global  tinlfoem  ultimate  boundedness  within  B(R) .  Given  any  bound 
reR^  there  exists  T(r)eR^  such  that  if  c ( • ) :  ( tj^)"*R^  Is  a  solution  of 
(Al.l)  with  Jc^llsr,  then  c(t)eB(R)  for  all  tat|j+T(r). 

The  listed  properties  of  the  solution  <(•)•’[ tj^)-»R*  can  bo  stated 
with  the  aid  of  the  following  lemma  (for  the  proof  see 
Corless'Laltmann,  1981). 
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Laaaa.  Given  system  (Al.l)  suppose  4P(0,C)>0  for  ell  CeR.  If  there 
exists  e  function  L  defined  on  {<{j2s  end  tE&,  end  If  there  exist 
dess  XR  functions  end  x^  end  e  dess  K  function  x,  such  thet 

x/||cI)SL(«,c)SXj(l<l|).  (Al.2e) 

|^L(e.t)+V*L(<.t)  S  -x,(i‘l).  (Al.2b) 

then  for  ell  y<||2s  end  t£R  the  system  trecks  the  reference  model  to 
within  eny  spherlcel  neighbourhood  B(R)  of  f>0  with  ■ 


Appendix  2 


Proofs  of  Thocrmms  1  end  2. 

Consider  es  Lyepunov  function  cendidete  for  system  (3.8)  with  the 
feedbeck  control  (3.11)  the  following 

V(«,)  -  (A2.1) 


Eveluetlng  the  derlvetlve  elong  the  solutions  of  the  closed  loop  system 
by  virtue  of  (3.9),  (3.10),  (3.11),  (3.12)  end  Assumption  S,  we  heve 


(l/2)V(«  )  -  -(i/2)t^q  f  -7  f^p  5  rp  f 


S  .(l/2)f|Q^f^.  7.|b;p^<.|*(1-«,)  + 

♦  |?P^<,|  |Hj(x^)ft^-Hj(x,)(i^+dj(x^)  1 
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(A2.2) 


+  llB^P.«JI|H^(x,)+Kj|!?J| 

+  llB^P.eJI  iH,(x,)x^-H^(x^)a +d^(x^)  II 

I  * 

S  (l/2)(-vJ|fJ|^  vJUJI  +  vj 
where  v  -  A  (Q  )  .  v  -  2k.  S  ,  and  v  -  2k  f  . 

1  ain  «  2  •  3  * 


AC  this  stage  the  application  of  Che  lemma  reported  in  Appendix  1 
proves  the  statement  of  the  Theorem  1. 

The  proof  of  Theorem  2  proceeds  exactly  in  the  same  way.  Ue  define  as 
Lyapunov  candidate  for  system  (4.5)  subject  to  Che  feedback  control 
(4.10) 


W(r,)  S 


(A2.3) 


The  derivative  along  the  solutions  of  the  closed  loop  system, 
considering  x  constant  in  the  fast  time  scale,  can  be  proved  to  satisfy 
the  following  inequality 

w(rj)  s  +  Wjlk,!l  +  «,  (a2.<*) 

with  w^S  A^__(Q^),  w^^  2«^k^,  and  w^S„;  +  w;||f||  ^  2i^k_^  +2«^k^lC||. 
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Appandlx  3 


Proof  of  Theorem  3 . 

The  proof  of  che  theorem  Is  based  on  the  combined  use  of  two  Lyapunov 
functions,  one  for  each  component  of  the  model  reference  tracking 
error. 

For  the  first  component  we  can  write 


f 


A^^(x)x+A^j(X)z-7,B^(x)B‘P^C-7^B^(x)B‘P^r 


+  7  B  (x)B^P  k  ^+a  (x)-&  X-&  2-S  u 

£  l  2  f  22  21  I  U  12  1 


-  (F^f-7  5?P^f-7,FE^(x)rP^f+Bj[H^(x)+K^]?+ 

+BJH^<x)^H^(x)C[+d^<x)  1 ) 

♦A^,(x)  l+BjC^j(x)  ) 


(A3.1) 


The  terms  within  braces  are  exactly  the  same  as  In  the  slow  model 
(3.8),  taking  apart  the  substitution  of  x  and  u  with  x  and  u.  On  the 

>  s 

basis  of  Assumptions  S  and  6,  and  recalling  (A. 10c),  It  Is  possible  to 
find  constants  a  ,  i-1,3  and  a  ,  1-1,3  such  that 

i  i 


-7  B,(x)rP  f] 

«  1  2  22  21^  * 

1  -  (x-r(i.S)  1 5 


(A3. 2a) 


ll«ll  -  |A^^(x)x+A^j(x)*.7^B^(x)BTp^C-7jBj(x)Bjp^r 

-  7  B  (x)B*P  & 'A  J+a  (x)-k  X’k  z-fi  uQ 
'f  1  2  f  22  K  '  J1  22  2  " 


(A3. 2b) 
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For  the  second  component  of  the  model  tracking  error  we  simply  rewrite 
equation  (4.8)  as 


+  BJG^(x)-7^(l^+E^<x))?PJf 

+  B^[G^(x)x+C^(x)z^+G^(x)u^-6^u^+d^(x)]  .  (A3. 3) 

Consider  the  function 


W(f.f)  ^  1/2  (A3. 4) 

and  evaluate  the  derivative  along  the  solutions  of  the  closed  loop 
tracking  error  system  (A3.1),  (A3. 3).  One  obtains 


w(«.r)  - 


-(;r  ■ 


(A3. 5) 


Consider  now  the  function 
V(0  £  1/2 


(A3. 6) 


and  evaluate  the  time  derivative  along  the  solutions  of  the  closed  loop 


Ill 


tracking  arrot  system  (A3.1),  (A3. 3).  In  view  of  (A3. 2a).  and 

conditions  (S.2)  and  (5.3).  wa  have 

V(f)  -  -vJ«||*+vJU||+v^+2iPjlfl[a 

a  -<v.-2aj|pjl)ifl^  +  bjfl  +  +  v^. 

(A3. 7) 


Wa  can  choose  as  Lyapunov  candidate  for  the  closed  loop  tracking  error 
system  (a3.1),  (a3.3)  the  following 


L(f.O  - 


^  22  21^ 


T 

P(c) 


(A3. 8) 


where 


P(c)  - 


r(i-c)p 

t 

0 


,  Oscsi. 


(A3. 9) 


In  view  of  (A3. 5)  and  (A3. 7)  the  time  derivative  of  (A3. 8)  along  the 
solutions  of  the  closed  loop  Cracking  error  system  satisfies 


LCf.n  s 


U\\ 

.A-  lA 


M(c) 


**  22  21 


+  m  <c) 


M 


+  *, 


where 


(A3. 10) 


M(c)  S 


■(l-c)(v^-2oJPj) 
.-l/2(caj+  (l-c)b^ 


-l/2(cej+(l-c)b^ 

/W 

•*)  . 


(A3. 11a) 


(A3. lib) 


B(C)  ^  [(1-C)b+C^ 

and 

w' 

m  -  V  (1-c)  +  c—  .  (A3. 11c) 

3 


V 

Provided  chat  guaranteed  by  condition  (5.4)),  the 

upper  bound  /j  of  parameter  ft  which  guarantees  the  definite  positivity 

P 

of  matrix  M(c)  is  given  bv  (see  Saberi-Khalil ,  1984) 


*‘p  (v^.2aJpj)aj  +  a^b^ 


(A3. 12) 


Chosen  ft  «  min(M,><).  for  each  0</j</i*  the  application  of  the  lemma 

p 

contained  in  Appendix  1  completes  Che  proof  of  Che  Theorem. 
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Abstract 

We  define  the  playability  property  of  a  qualitative  differential  game,  and  we 
characterize  it  by  a  regulation  map  which  aaeociates  with  any  playable  state  a 
set  of  playable  controls.  We  extract  among  theses  playable  controls  the  set 
of  discrindnating  and  pure  controls  of  one  of  the  players.  We  characterize 
them  through  an  adequate  "contingent’'  Hamilton- Jacobi-Isaacs  equation, 
and  we  provide  sufficient  conditions  implying  the  existence  of  continuous  or 
minimal  playable,  discriminating  and  pure  feedbacks. 


R48um4 

Nous  ddfinissons  une  propridtd  de  jouabiiitd  de  jeux  difflfrentiels  qualitatifs, 
que  nous  caractdrisons  i  I’aide  d’une  correspondance  de  rdgulation  qui  as- 
socie  k  tout  dtat  jouable  un  ensemble  de  contrdles  Jouables.  On  distingue 
parmi  ces  contrdles  jouables  I'enaemble  des  coBtrfiles  discriminants  et 
des  contrdles  purs  d’un  des  joueurs.  Nous  caratdrisons  ces  concepts  par 
une  Equation  d’Hamilton- Jacobi-Isaacs  "contingente*,  et  nous  dnon;ons  des 
conditions  sufiisantes  impliquant  I’exiatence  de  retroactions  jouables,  dis- 
criminantes  et  pures. 
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We  conaider  a  two-player  differential  game  whoae  dynamics  are  described  by 

0  *'(0  =  /(*(0.V(‘)i“(0) 

“>  if)  u(t)  €  U{x(t),y{t)) 

M  0  y'(‘)  =  »{*W.»(0.«(*)) 

ii)  v(t)  €  v(*(t),y(t)) 

The  roles  of  the  game  are  set-valued  maps  P  :Y  ^  X  and  Q  :  X 
Y,  stating  the  constraints  imposed  by  one  player  on  the  other. 

The  playability  domain  of  the  game  C  X  x  V  is  defined  by: 

^  ~  {  (ii  y)  £  X  X  K  I  I  €  P(y)  and  y  €  (?(z)  } 

(We  conaider  only  the  time-independent  case  for  the  sake  of  simplicity). 

The  playability  property  states  that  for  all  initial  state  (zo,y&)  £  X, 
there  exists  a  solution  to  the  differential  game  which  is  playable  in  the  sense 
that 

Vf>0,  x(t)  €  P(y(<))  &  y(t)  6  Q(i(t)) 

We  shall  charaterize  it  by  constructing  decision  rules 

(x,y,v)^*(x,y;v)  te  (x.y.u)  *(x,y;tt) 

which  involve  the  contingent  derivatives^  of  the  set- valued  maps  P  and  Q, 
with  which  we  build  the  regulation  map  P  mapping  each  (x,  y)  €  X  to 
the  regulation  set 

X(*.y)  =  {  (u.v)  I  «€*(», y:v)  and  v€  ♦(x,y:u)  } 

The  controls  belonging  to  P(x,  y)  are  called  playable. 

‘We  recall  that  the  eoBtlaseat  cone  rjr(x)  to  a  sobeet  X  at  a  €  X  is  the  cloeed  cone 
of  elements  e  satisfying 

UmU  d(z  +  ho,  X)/h  =  0 

The  coDtiageat  derivative  of  the  set-oalned  map  Q  from  X  to  T  at  a  point  (s,  y)  of  its 
graph  is  the  cloesd  poeithely  homogeaoas  set-vahied  map  P(7(a,  y)  from  X  to  Y  detned 
by 

Graph(OQ(a,y))  :»  Tcr^htoi^*’*') 

or,  equivalently,  by. 

ee0Q(z,y)(o)  Urn  inf  dfe, 

s— e+.v'— •  \ 
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The  Playability  Theorem  states  that  under  technical  asnunptions,  the 
playability  property  holds  true  if  and  only  if 

V(*,y)€if,  R(x,y)  ^  0 

and  that  playable  soluticms  to  the  game  are  regulated  by  the  regulation 
law; 

Vt>0,  u(t)  €  ♦(*{t),y(t);v(t))  ic  v[t)  6  ♦(i(t),y(t): u(t)) 

We  then  introduce  the  subset 


Aiz,y;v)  :=  {  u  €  (x,  y)  |  (u,  v)  €  y)  } 


of  discriminating  controls  which  allow  the  first  player  to  associate  to  any 
control  V  €V(x,  y)  played  by  the  second  player  at  least  a  control  u€U{x,  y) 
such  that  the  pair  (u,  v)  is  playable  and  the  subset 

:=  n 

•Cl'Cejr) 


of  pure  controls  which  allow  the  first  player  to  find  a  control  u€  U(x,  y) 
such  that  (u,  u)  is  playable  for  all  v  6  V(x,y). 

These  concepts  are  particularly  relevant  for  games  ‘against  nature”  or 
‘disturbances” (see  1 11, 12, 26, 27]  and  their  references). 

Before  going  further,  it  nwy  be  useful  to  relate  these  concepts  to  more  fa¬ 
miliar  ones  through  an  adequate  Hamilton-Jacobi-lsaacs’s  equation  (8ee[18j). 

For  that  purpose,  we  characterize  the  rules  P  and  Q  by  their  indicator 
functions  Wp  and  Wq  defined  respectively  by 


Wp{x,y) 


0  if  z  e  P(y) 
-boo  if  i^P(y) 


*»'<j(*.y) 


0  if  y€<J(z) 
-boo  if  y^Q(z) 


These  functions  are  only  lower  semkontinuous,  but  we  can  still  ‘differ¬ 
entiate*  them  by  taking  their  contingent  epiderlvatlves*.  We  set 


ff(Wp  +  Wq;x,y;u,v)  :=  Dj(Wp +  1Vq)(x,y)(flx,y;u),f(x,y;v)) 

’The  coatiafmt  derivative  DiW(a)  et  a  oteaded  fuetioB  W  bom  X  to  R  u  {-foo} 
at  s  €  DomflF)  is  deSned  by 

SpD,fr(t)  rr,w')(s.W(s)) 

or,  eqnhaleatly,  by 
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We  sheU  prove  that 

—  the  came  ia  playable  if  and  only  if 


{ 


and  the  reculation  map  ia  equal  to 
^(*.y)  =  {(“.«)  €  y(*,y)  X  V(x,y)  I 

B{Wp  +  W<}\x,y;u,v)  =  inf„igp(,.»).,<6V(*,i,)  B{Wp  +  WQ-x,y,u',v')) 
—  the  fint  player  haa  a  diacriminating  control  if  and  only  if 


aup  inf  B(Wp  +  Wo-,x,y;u,v)  =  0 

and  the  feedback  map  A  ia  equal  to 
/  A(i,y;«)  =  {tt€ff(i,v)| 

\  B{Wp  +  W<);x,y;tt,o)  =  iafu'eVM  B(Wp  +  WQ;x,y;u' ,v)} 

—  the  firat  player  haa  a  pure  control  if  and  only  if 

inf  aup  £r(Wp  +  lVq;x,y;u,v)  =  0 

and  the  feedback  map  B  ia  equal  to 

f  B(x,y)  =  {u€  ff(*,y)  |aup,6v(,,)ff(W;>  +  WQ;*,y;u,v) 

\  =  inf«<g(r(x,y)  aup^v(,_,j  ir(Wj.  +  W<j;  x,  y,  u',  u)} 

We  then  deal  with  the  main  topic  of  thia  paper;  conatruct  single-valued 
playable  feedbacks  (u,  v),  such  that  the  differential  system 

/  xf{t)  =  /(*(<).y(<).fi(*(t).»W) 

\  **(0  =  »(»{0.v(*).*(*{0.v(0) 

haa  playable  solutions  for  each  initial  state.  By  the  Playability  Theorem, 
they  most  be  selections  of  the  regulation  map  R  in  the  sense  that 

V  (x.y)  €  if,  (x.v)  (tt{a,y),v(x,y))  e  R{x,y) 

We  shall  prove  the  existence  of  such  continuous  single-valued  playable 
feedbacks,  as  well  as  more  constructive,  but  discontinuous,  playable  feed¬ 
backs,  such  as  the  feedbacks  associating  the  controb  of  R[x,y)  with  mini¬ 
mal  norm  (the  playable  alow  feedbacks,  as  in  |13,96]  ).  More  generally,  we 


X21 


shall  show  the  existence  of  possible  set-valued  feedbaclu  associating  with 
*i>y  (Si  si)  ^  ^  of  controls  (u,  v)  6  R{x,  y)  which  are  solutions  to  a 

(static)  optiinisation  problem  of  the  form: 

(u, »)  €  R{x,  y)  I  <r(* ,  y;  u, »)  <  ^  v: v') 

or  solutions  to  a  noncooperative  game  of  the  form: 

V{o',  v')  €  R(,z,  y),  a(z,  u, «')  <  o(x,  u,  v)  <  o(z,  u',  v) 

In  other  words,  the  players  can  Implement  playable  solutions  to 
the  differential  game  by  playing  for  each  state  (z,y)  £  /T  a  static 
game  on  the  controls  of  the  regulation  subset  R(x,y). 

We  also  consider  the  issue  of  finding  discriminating  feedbacks,  which 
are  selections  of  the  set-valued  map  44.  We  shall  provide  for  instance  suffi¬ 
cient  conditions  implying  that  for  all  continuous  feedback  tf(z,y)  e  V{x,y) 
played  by  the  second  player,  the  first  player  can  find  a  feedback  (continuous 
or  of  minimal  norm)  tt(z,y)  such  that  the  above  differential  equation  has 
playable  solutions  for  each  initial  state. 

Finally,  we  address  the  question  of  constructing  continuous  pure  feed¬ 
backs  S(x,y)  which  have  the  property  of  yielding  playable  solutions  of  the 
above  differential  equation  whatever  the  continuous  feedback  v(z,  y)  played 
by  the  second  player*. 

We  use  for  constructing  these  feedbacks  selection  theorems  (for  instance, 
Michael's  continuous  selection  theorem,  see  [29,30,31]),  we  need  to  prove  the 

*Oa«  csa  also  coastmct  Myasmie  fsedbaefc  controls*  wUch  art  aalactioiu  (i,  c)  of  tho 
eoatiafeat  dorivatlva  of  tho  rofolatioa  nap 

(i(s,  r.  V.  V),  f(s,  t : v))  €  DK{X,  ,)(/(x,  r, «).  g{*.  r,  •)) 

With  thoao  ’dynamic  foadbacka,  pUyora  Implamant  tho  digonatial  ayatan 

I  ,'(f)  «  t(x(thmi(*(t),r(f)) 

[  ,'(t)  -  f(.(t),,(t)i*(0.v<t)) 
which  yields  pUysbIo  solatloaa. 

la  other  words,  the  pUyeri  can  tanplemsat  pUynble  aohitieiu  to  the  differential 
gnma  by  playlag  for  each  state  (s,ir)  €  K  a  atatie  game  on  “velocities*  of  the 
ecotreb  in  the  derivative  0/{(s,y)(/(x,y;«),f(s,y;v))  of  the  ragaJatiea  aiibaet. 

Mlaistal  aeleetloas  (h*,a*)  paovide  heavy  trajectories  (sea  |$|)  la  the  case  of  coatrol 
syatoau 


lower  semicoiitiiiuity  of  the  eet-valued  mape  Jt,  A  and  B.  In  the  case  of  the 
set-tralued  map  B,  we  need  a  Lower  Semicontinuily  Criterion  of  an  infinite 
intersection  of  lower  semicontinuons  maps.  We  provide  such  a  theorem  at 
the  end  of  this  paper,  which  can  be  us^l  for  other  purposes. 
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Abstract 

A  dynamic  decision  model  is  said  to  be  forward-looking  if  the  evolution  of  the  underlying 
system  depends  explicitly  on  the  expectations  the  agents  form  on  the  future  evolution  itself. 
Such  models  lead  to  nonstandard  stochastic  dynamic  optimization  problems  where  one  has 
to  take  into  account  the  fact  that  there  is  a  circulu  (closed)  relationship  between  future 
forecasts  and  future  system  behavior.  In  this  paper  we  study  a  class  of  such  problems  where 
there  is  an  additional  control  input  designed  to  make  the  system  track  a  ^ven  trtoectory. 
This  leads  to  a  game-theoretic  formulation  in  which  context  we  consider  both  finite  and 
infinite  horizon  formulations.  It  is  shown  that  for  the  finite  horizon  problem  the  tinique  Nash 
equilibrium  solution  requires  (fixed  size)  memory  for  both  agents  because  of  spillover  across 
stages,  whereas  for  the  infinite  horizon  version  no  memory  is  needed. 

1.  An  Introduction  to  Forward-Looking  Models 

We  refer  to  a  dynamic  stochastic  model  as  forward-looking  if  one  of  its  inputs  involves 
future  expectations  of  the  system  trajectory,  using  (possibly  noisy)  measurements  on  the  past 
realizations.  Such  decision  models  find  wide-spread  use  in  economics,  where  they  are  more 
commonly  known  as  rational  espertations  models.  A  few  representative  papers  in  this  area 
are  the  works  of  Lucas  (19T5),  Sargent  and  Wallace  (1975),  Barro  (1976),  Taylor  (1977), 
Sbiller  (1978),  Blanchard  (1979),  and  Blanchard  and  Kahn  (1980).  Perhaps  the  simplest 
such  model  that  captures  the  salient  features  of  forward-looking  behavior  is  described  by  the 
scalar  difference  equation 


Vt+i  =  m  +  bvt  +  ft+i,  (la) 

where  a  and  b  are  constant  parameters,  {ci}  is  a  sequence  of  independent  zero-mean  random 
variables  with  finite  (positive)  variance,  and  vt  is  the  decision  variable  chosen  at  time  t  under 
some  "expectation’'  of  the  future  behavior  of  the  system  based  on  information  available  at 
time  t.  If  the  forecast  of  interest  is  n  steps  into  the  future,  for  example,  one  possibility  is  to 
replace  vt  in  (la)  by  Etyt+n,  the  conditional  mean  of  pt+n  based  on  the  information  available 
at  time  t.  This  information,  which  we  denote  by  qt,  could  involve  a  direct  measurement  of 
all  the  past  values  of  the  system  trajectory,  that  is  {yt,yt-t,—}  =:  V*f  or  involve  some  noisy 
measurement  on  the  state  triqectory,  tjt  =  m*,  where  {z(}  is  a  measurement  sequence  defined 
by 

1  This  work  was  performed  while  the  author  was  spending  a  sabbatical  year  at  INRIA, 
Sophia  AntipoHs,  Fttmee,  and  it  was  also  partudly  supported  by  the  Air  Force  Offiea  of  Sci¬ 
entific  Research  under  Grant  No.  AFOSR  084-0056,  through  the  University  of  Rlinois. 
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Mt  —  yt  +  it,  (16) 

with  {((}  being  another  sequence  of  indep«i<ient,  zero-mean  random  variables  with  finite 
variance. 

A  basic  question  addressed  in  the  literature  over  ^  years  has  been  the  existence  of 
a  (unique)  stochastic  process  {pt}  that  satisfias  (la)  whenever  ot  Etpt+n  and  the  time 
interval  is  infinite.  The  answer  to  this  question  is  that  there  is,  in  general,  more  than  one 
such  solution  even  in  the  class  of  stationary  processes.  However,  as  we  have  recently  argued 
in  Boyar  (1987),  a  better  ^proach  towards  policy  determination  in  these  forward-looking 
models  would  involve  the  optimization  of  an  appropriate  loss  function,  by  carefully  taking 
into  account  the  informational  dependence  as  well  as  the  correlation  of  policies  across  stages. 
One  such  criterion  would  be 

Jj  :=  min  53  ^?{H«(»Jt)  -  y»+«]*}p*"*,  (2) 

where  minimization  is  subject  to  the  dynamic  constraint  (la),  with  vt  =  7t(q<),  and  uses 
the  boundary  condition  ««  s  0  for  t  >  T.  In  the  above,  [s,r]  stands  for  the  time  horizon, 
which  could  also  be  infinite,  and  p  denotes  a  positive  discount  factor  (0  <  p  <  1).  It  has  been 
shown  in  Ba^ar  (1987)  that  the  dynamic  policy  optimization  problem  admits  the  solution 
Vi  =  Etiit+i  when  n  =  1,  but  for  n  >  2  the  unique  solution  for  the  finite-horizon  version  is 
different  from  Etyt+n-  For  n  =  2,  for  example,  the  best  forecast  into  the  future  (by  two  time 
steps),  under  the  criterion  (2)  and  using  the  information  s  y*},  is  given  by 

"t  =  iHv*)  =  ««yt  +  fitvt-u  (3o) 

for  2  <  t  <  T,  where  the  sequences  {««}  and  {fit}  are  determined  recursively  off-line.  For 
the  noisy  measurenunt  case,  {vt  =  s*},  the  solution  is  again  unique  and  is  given  by 

Vf  =  Iti**)  —  +  0t^t-i,  (36) 

for  2  <  t  <  T,  where  the  sequences  {at}  and  {A}  are  the  same  as  in  (3a),  and  yt  is  a 
sequence  of  estimates  generated  recursively  by  a  Kaltnan  filter,  under  the  assumption  that 
the  underlying  statistics  are  Gaussian.  An  interesting  feature  of  the  solution  is  that  for  the 
infinite-horizon  version  (that  is  as  T  -»  oo)  the  coefficient  sequence  {fit}  vanishes  for  all  finite 
(,  and  the  solution  becomes  =  Biy(.|.3,thus  eliminating  the  correlation  across  stages. 

In  the  present  paper,  we  consider  a  man  general  formulation  than  that  above,  where 
now  two  separate  agents,  say  A  and  B,  have  influence  on  the  system  trajectory,  one  of  them 
(A)  again  making  a  two-step  ahead  forecast  of  the  trajectory,  whereas  the  other  one  (B) 
trying  to  drive  the  trajectory  as  close  to  a  specific  target  as  possible.  For  such  a  scenario,  the 
system  equation  would  be  replaced  by 

y<+i  »  oy# -f  6of  +  cwf  + (4) 

where  vt  —  'iti.y*)  Is  controlled  by  agent  A  and  wt  =  l<«(y‘)  by  agent  B.  Taking  the  time 
horizon  as  [0,r  -t- 1],  the  two  cost  functions  to  be  by  A  and  B,  respectively,  are 

/a(7,a*)  =  53^{h»(y‘)  -  y»+zl*}Px. 

taO 


(5a) 


and 


(54) 


Jailift)  =  53  “  ft+il*  +  }^B> 

imO 

where  {j7t,  3<t<T+2}is  the  deeired  trajectory,  h  is  a  poeitive  weight  on  agent  B’a  control, 
Pa.  Pb  are  the  corresponding  diacount  factors,  7  :=  {'yr.TrT-i.-.Tro},P  :=  {mt+i.Mt,..,po}. 
and  vr-fi  =  0,  the  last  identity  reflecting  the  fact  that  no  forecast  is  made  at  time  t  =  T  +  l. 
Furthermore,  we  assume  that  the  independent  random  variables  u  ll<t<T+2)  each  have  zero 
mean  and  a  probability  distribution  that  assigns  positive  probability  to  every  measurable  open 
subset  of  the  real  line.  One  such  (Ustribution  would  be  the  normal  (Gaussian)  distribution 
with  positive  variance. 

Since  this  is  a  problem  with  multiple  objectives,  several  equilibrium  solution  concepts 
would  be  applicable,  with  the  one  adopted  here  being  the  noncooperative  Nash  equilibrium 
solution.  Therefore,  we  seek  a  pair  (7*,^*),  preferably  unique,  satisfying  the  pair  of  inequal¬ 
ities 


./a(7*.P*)  <  Ja(7,P*);  Jb(7*,M*)  <  Jah*,!*),  (6) 

for  all  admissible  7  and  tx.  Other  possibilities  would  have  been  the  Stackelberg  solution  with 
either  agent  acting  as  the  leader  and  the  Pareto-optimal  solution,  which,  however,  we  do  not 
discuss  here  because  of  space  limitations. 

The  first  question  we  attack,  in  section  3,  is  a  “simpler”  version  of  the  above,  where 
agent  A’s  policy  is  fixed  as  vt  =  Etyt+i,t  <  T,  which  is  in  general  a  suboptimal  policy  for 
A.  We  obtain  the  best  policy  for  B  under  this  additional  structural  restriction,  and  derive 
the  corresponding  expression  for  {m<}  (mc  Theorem  J).  Furthermore,  we  study  the  limiting 
behavior  of  the  two  policies,  for  the  infinite-horizon  problem.  Subsequently,  in  section  3, 
we  derive  the  unrestricted  Nash  solution  and  prove  its  (generic)  existence  and  uniqueness 
(see  Theorem  2),  with  detrils  of  the  derivation  provided  in  the  Appendix.  We  also  study 
the  limiting  behavior  of  the  solution  as  T  -*  00,  and  analyse  the  discrepancies  that  exist 
between  the  two  stationary  solutions  of  Theorem  J  and  Theorem  2.  The  paper  concludes 
with  a  discussion  of  the  “noisy  measurement”  case  and  some  other  possible  extensions,  in 
section  4.  Throughout  the  analysis,  we  take  the  reference  trajectory  (to  be  tracked)  as  the 
sera  trajectory,  an  assumption  that  does  not  bring  in  much  ioes  of  conceptual  generality  but 
leads  to  considerable  simplifications  in  the  resulting  expressions. 

2.  The  Optimal  Tracking  Strategy  Under  Perfect  Myopic  Forecast 

With  vt  taken  as  EtVt+2  (which  myopically  minimizes  each  term  of  {5a)),  and  {f/t} 
taken  as  the  sera  trajectory,  the  dynamic  policy  optimization  problem  faced  by  agent  B  is 
the  minimization  of  where 


T+l 

+  kwf}p*s\  (7a) 

tmt 

the  dynamic  constraint  is 

VT+a  =  «Vt+i  +  ewT+i  +  *r-n. 

vt+i  ®  ®v»  +  4^iv«-(-3 +  «wt  +  *t+i. 

and  the  information  constraint  is  wt  »  Mt(y*}-  ff®te  that  this  is  not  a  standard  linear- 
quadratic  stochastic  control  problem  because  of  the  presence  of  the  conditional  expectations 
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tana  in  (76),  which  could  even  make  the  dynamic  eonatraint  nonlinear  in  the  paat  valuea  of 
the  trajectory.  We  will  ahow  below,  however,  that  the  optimal  control  ia  atill  linear,  thua 
making  the  correapondiag  forecaat  alao  linear  in  the  available  information.  The  derivation 
entaila  a  recuraive  approach  where  the  atmcture  of  «t  determined  alongaide  the  optimal 
control  at  each  atep  of  the  iterative  minimiaation. 

Before  presenting  the  main  result  of  this  sectbn,  we  first  introduce  two  sequences  {pt} 
and  {>/(}  which  ere  defined  recursively  by 


pt  =  1  +  -3- — rrrs*  Pr+2  - 1. 


abkut 

Next  we  define  a  third  sequence  {gt}  in  terms  of  the  other  two,  according  to 


(8a) 

(86) 


gt  =  — capt+i/(c*pt+i  +  kt/f),  t  <T  +1.  (8c) 

We  are  now  in  a  position  to  state  the  main  result,  after  invoking  a  condition  which  generically 
holds. 

Condition  1.  The  sequence  {(/«}  generated  by  (86)  does  not  vanish  for  any  t  <  T  +  1. 

Theorem  1.  Let  Condition  1  be  antisSed.  Then,  the  dynamic  policy  optimisation  problem 
with  myopic  forecaat  admits  the  unique  solution 


v>t  =  A«(»t)  =  gm,  0  <  t  <  r  +  1, 

with  the  corresponding  forecast  policy  given  by 

,,  =  =  («±£££il){±±££0 

The  minimum  value  of  in  (6a)  ia 

Fq'*'^  =  Po^iVo}  +  Ao, 

where  Ao  is  the  last  step  of  the  backward  recursion 


(9a) 


(96) 


(9c) 


Ar+i  =  »ar{«r+a)t 

A«_i  ®  PB^i  +  Ptvor(et). 

Proof.  The  proof  proceeds  by  recursively  showing  that  the  Tninimnm  value  of  F^*^  is  given, 
for  euh  s  <  T  +  1,  by 


^■f-"‘  =  l(p.-i)/PB!£{y;}  +  A.. 

The  result  is  trivially  true  for  s  s  T  +  2,  where  we  take  Ar.t-3  =  0.  Let  us  therefore  assume 
its  validity,  along  with  (9a)  and  (96),  up  to  s  1,  and  verify  the  expression,  as  well  as  (9a) 
and  (96),  for  a.  The  minimisation  probiem  faced  by  agent  B  at  time  s  is 

+  *i*.(v*)*}].  (•) 
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which  is  equivalent  to 

+  hu»*  I  V*}, 

10 

which  uses  the  dynanuc  constraint 

y.+i  =«»«  +  6^*y*+a +  «"  +  «»+!  •  {*) 

We  also  have  the  relationship 

y*+a  =  (a  +  ey«^t)y<^.i  +  6((tt  +  ey«^])(a  +  cy«-t-i)/i'«-i-at'*-i-iIy«-(-i  +  *»+a> 

where  we  have  explicitly  \ised  (9a)  and  (96),  with  t  replaced  by  s  + 1.  (Of  course,  if  s  =  T+ 1, 
the  last  relationship  would  not  be  needed  since  the  conditional  expectation  term  in  (*)  Would 
be  missing.)  Now,  taking  the  conditional  expectation  of  the  last  expression  with  respect  to 
substituting  this  into  (v),  taking  the  conditional  expectation  of  the  resulting  expression 
again  with  respect  to  y*,  and  solving  for  the  resulting  Ety,+i  in  terms  of  y«  and  to,  we  arrive 
at  the  expression 

E,yt+t  =  — [«y*  +  ««’]• 

Using  this,  E,yt+2  can  easily  be  evaluated  to  be 

„  a  +  egt+i  +  6h,^.i ,  .  ,  ,  , 

E»yt+2  - - - - [ay,  +  eto],  (**) 

under  which  the  dynamic  constrsdnt  becomes  equivalent  to 

y.+i  =  ^|ay«  +  cto]  +  €,+i. 

•'0 

This  makes  the  minimization  problem  a  standard  Unear-qvadratie  one,  and  hence  it  readily 
follows  that  the  minimizing  control  is  uniqtiely  given  by  (9a)  with  t  =  s.  Substitution  of  this 
solution  into  (•)  and  (**)  finally  verifies  the  asserted  form  for  and  the  structure  of  the 
forecast  policy  as  given  by  (96).  We  should  note  that  Condition  1  has  explicitly  been  used  in 
the  proof,  to  make  sure  that  one  can  solve  uniquely  for  E,y,^\  and  Efy,+2-  o 

Condition  1,  under  which  the  existence  and  uniqueness  of  the  optimal  control  (9a)  is  valid, 
holds  whenever  a  and  6  have  opposite  signs,  r^ardless  of  the  magnitudes  of  the  parameters  of 
the  problem.  The  result  follows  by  Inspection,  since  with  a6  <  0  and  vr+i  *  1,  we  have  i/t  >  0 
for  all  t  <  r  +  1.  For  a6  <  0,  however,  there  may  exist  isolated  values  for  the  parameters 
for  which  the  condition  does  not  hold  for  some  t.  [A  more  precise  statement  here  would  be 
that  with  all  but  one  of  the  parameters  fixed  (and  a6  >  0),  there  will  exist  at  most  a  finite 
number  of  difierent  values  of  that  parameter  for  which  Condition  1  is  violated.  This  follows 
since  for  each  t,  i/t  is  a  rational  function  cd  the  quintuplet  (a,6,e,pa,ik).]  For  example,  for 
the  parameter  values  aacsfcasi,  6*2,  we  have  a  0,  which  shows  that  Condition  1 
may  fail  even  for  a  two-stage  problem.  However,  if  we  perturb  the  value  of  6  to  6  s  2.1,  and 
take  PB  1,  then  Condition  1  holds  for  ail  values  of  t.  In  fact,  running  the  coupled  recursive 
equations  (8a)>(86)  in  retrograde  time,  we  find  that  (for  these  parameter  values)  the  pair 
(i/t,P()  converges  to  (0.504147, 1.880960)  in  29  steps,  within  ths  accuracy  of  six  decii^  places. 
Hence,  in  this  case,  the  infinite-horizon  version  (even  with  no  discounting)  admits  a  unique 
optimal  stationary  control,  given  by  wt  »  fl(v$)  ™  —1.135124  yt.  If,  in  the  above,  6  is  instead 
taken  to  be  1,  again  Condition  1  holds,  the  pair  (vi,P<)  convargas  to  (0.694146, 1.787692)  in  9 
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itantions,  and  the  optimal  control  policy  convergM  to  =  A(vt)  —  —0.787692  yt.  As  a  final 
tinmmcxl  cgcpcrimentation,  refiecting  a  different  set  of  parameter  values,  we  consider  the  case 
of  a  ao  2,6  s  —Zu>B  =  0.8,e  s  h  »  1.  For  this  set,  we  already  know  that  Condition  1  holds, 
since  a6  <  0.  Studying  the  convergence  of  the  optimal  policy  to  a  stationary  control,  we  find 
that  the  pair  {ut,pt)  converges  to  (2.796267,1.521150)  in  26  iterations,  with  the  resulting 
stationary  policy  being  to«  «  —0.325719  yt. 


S.  The  Nash  Hquilibrium  Solution 

We  now  remove  the  restriction  that  agent  A’s  input  to  the  system  is  a  nqropic  forecast, 
and  allow  hitn  to. determine  the  *best”  choice  for  V(  by  minimising  the  cost  function  Ja- 
As  we  have  discussed  in  aeetion  1,  this  joint  optimisation  problem  can  best  be  treated  as 
a  ttoncooperathre  game,  and  hence  we  study  in  this  section  the  Nash  equilibrium  of  the 
underlying  game,  as  defined  by  (6). 

There  are  two  general  ^preaches  to  the  derivation  of  Nash  equilibria  in  such  dynamic 
games.  One  would  be  first  to  guess  (or  propose)  a  structure  for  the  solution  in  terms  of 
some  parameters,  and  then  to  validate  the  equilibrium  property  of  the  asserted  structure 
and  to  obtain  the  corresponding  values  of  the  parameters  so  that  the  resulting  policies  are 
in  Nash  equilibrium.  A  second  approach  would  be  to  obtw  the  Nash  solution  recursively 
(by  employing  the  definition  of  stagtwite  or  feedback  equilibrium;  see,  for  example,  Bagu  and 
Olsder  (1982))  by  solving  static  games  conditioned  on  the  avmlable  (common)  information, 
at  each  step  ^  the  iteration.  Note  that  this  would  be  applicable  only  if  both  agents  have 
identical  information  (which  is  the  case  here),  since  otherwise  stagewise  decomposition  would 
not  be  possible.  Now,  two  disadvantages  of  the  first  method  are  that  (i)  one  has  to  guess  the 
structure  of  the  solution  correctly,  and  (ii)  even  if  the  initial  guess  is  correct  there  is  no  way 
to  show  (using  this  method)  that  the  validated  Nash  solution  is  unique.  The  second  method, 
on  the  other  hand,  is  capable  of  answering  the  uniqueness  question,  but  it  only  produces 
candidate  solutions  which  subsequently  have  to  be  decked  for  their  equilibrium  property. 
What  we  will,  therefore,  chooee  to  do  in  the  sequel  is  to  use  an  appropriate  combination  of 
'the  two  approaches,'to  generate  candidate  solutions  and  verify  their  existence  and  uniqueness. 
We  should  note  in  passing  that  even  though  the  problem  may  look,  at  the  outset,  as  a  standard 
linear-quadratic  one,  the  presence  of  the  two-step  delay  in  the  coet  function  of  agent  A  makes 
the  game  a  nonstandard  one,  thus  eliminating  the  possibility  of  direct  ^plication  of  results 
available  on  linear-quadratic  feedback  Nash  games  (as,  for  example,  covered  in  Bagar  and 
Olader  (1982)), 

Before  presenting  the  solution  in  Theorem  2  below,  we  first  introduce  some  sequences 
which  will  be  needed  in  the  characterization  of  the  equilibrium  policies.  Towards  this  end, 
let  {mt},  {rhi},  {n*}  be  three  sequences  generated  by 


mt-i  = 


1  .  ~  ,  ok 

q4-eoe-t-6afmi]:  my  = 


1  —  6m« 


k-^c»' 


m«_l  : 


1  —  hmt 

_  Pa**i(l  -  6mt)^ 

*  *  6*  -f-  pa'H(1  “  ktn%)^ ' 

where  at,  St,  0i,  3t  we  defined,  for  t  <  T,  by 


[c0t  +  6(?imtJ;  my  »  0, 

»T  =  1, 


(lOe) 

(106) 

(lOc) 


flit : 


(1  -  hmt)pA*hmt  -  6 


6*  +  Pa«t(l  -  6mt)(l  -imt-mt) 


^-r{o+eStl,  1ST, 


(lie) 
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St-i  t-  (^u.« f<r  +  i, 

k  +  c>ntt.i 

(116) 

(1  -  6mt)p,tntmtefit  +  b(l  -  cfit)  *  ^  7- 

6*  +  PA^hii  ~  6mt)(l  —  6mi  —  mt)  ’  ~  ’ 

(12a) 

X  {bkll,t  +  kt3,t)efit-l  ^  n,  .  , 

- fcTPw  ’  -  ’ 

(126) 

and 

\*xa,t  *aa,t  / 

is  a  2  X  2  matrix  sequence  generated  by  the  discrete  time  Riccati  equation 

Jft  *  PaAJIJft+i  -  Kt+iC(C»Jift+iC  +  ife)-»C'Jfl  +  llAt  +  Q, 
Kt+1  -  lka^PB/(k  +  e*)lQ, 

(13a) 

with 

(136) 

Finally,  let  rot,  rat,  r/jt,  f^t  ^  <lefined  by 

c|(l  -  6mt)pH»»t»t  -  b) 

■*  6*  +  PAnt(l  -  6mt)(l  -  butt  -  mt)  ’ 

(14o) 

e(6fcix4+i  +  k2a,t+i) 

- - 

(146) 

c(l  -  bmt)pAntmt  -  6e 
'  6*  +  pxnt(l  —  bmt)(l  -  6mt  -  mt)  ’ 

(15a) 

«(6fcii,t+i  +  bia,t+i) 

'  b  +  e**u,t+i 

(156) 

Tha  last  four  expressions  are  the  coefficient  terms  in  (lla)>(126),  indicating  the  dependence 
of  Of,  S|,  fit  and  fit  on  St,  at,  fit  and  fit,  respectively.  A  certain  relationship  between  these 
coefficient  terms  in  fact  determines  the  existence  of  a  nnique  Nash  equilibrium  solution,  as 
to  be  eliicidated  below. 

Condition  3.  For  all  t  <  T, 

ratrn  #  1,  rfitrjft  /  1.  (1®«) 

tmtitl,  (166) 

-  6mt  -6*.  (16c) 

Theorem  3.  Let  Condition  3  be  satisfied.  Then,  the  forwerd-lookiug  tracking  model  (4)>(5) 
admita  a  unique  Naah  equilibrium  solution  {'it,itth  '"ffive  agent  A’s  (beet  forecast)  policy 
k 


■  'll  (»*)  ■  atvt  +  fiiVt-u  t  i  1 
-  oops,  t «  0, 


(17) 
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and  agent  B’s  (bt$t  tracking)  policy  is 

*  /*?(»*)  =  ««»*  +  Avi~u 

--[ac/(fc  +  e*)liKr+i, 

*  SoJtoi 

where  the  sequence  {««}  is  generated  by 

»«  =  =  +  fitvt-u  t  >  1  , 

=  otoW.  t  =  0. 

Proof.  We  will  &st  verify  the  etmctnral  cooeistency  of  the  eolation  (17)-(18)  under  the  Nash 
inequalities  (6),  and  then  discuss  the  existence  of  the  solution.  Some  details  of  the  derivation, 
as  well  as  a  proof  for  the  uniqueness  of  the  solution  will  be  given  in  the  Appendix. 

Towards  verifying  the  validity  of  (6),  first  consider  the  second  inequality,  where  agent 
A’s  policy  has  been  fixed  as  given  by  (17).  Then,  agent  B  faces  a  stochastic  control  problem 
with  cost  function  Jp  (given  by  (56)  with  zero  reference  trajectory)  and  state  dynamics 

V»+i  =  (o  +  6a()pt  +  6/JiSt-i  +  MOt  +  €t+i  ,  1  <  r 

=  oyr+i  +  cv/T+i  +  «T+j  »  t  =  T  + 1, 

where  the  sequence  {vt}  is  generated  by  (18b)  in  view  of  (17).  The  optimal  control  at  time 
T  +  1,  u>T+i,  can  readily  be  obtmed,  to  be  given  by  the  second  line  in  (I8a).  To  obt^ 
the  remiuning  controls,  we  introduce  a  new  state  vector,  zt  (vti  vt-i),  and  reformulate  the 
problem  as  one  of  minimising  Jb  under  the  dynamic  constraint 

**+j  =  A*z«  +  Cwf  +  Dii+i  ;  V  (1  0)', 

where  control  wt  is  allowed  to  depend  on  x*,  t  <  T.  [Note  that  even  though  is  not 
av^able  to  agent  B,  is  since  it  is  generated  by  y*'*.]  This  is  the  familiar  linear- 
quadratic  optimal  control  problem,  whose  unique  solution  is 

«,«  =  -{fc  +  C'Kt+iC)-^C'Kt+iAtXt,  (*) 

where  {/ft}  ■>  generated  by  (13o).  [Note  that  the  terminal  constraint  on  ift  at  t  T  +  1 
is  not  Q  bMuse  w«  have  already  substituted  for  the  optimal  and  have  reduced  the 
cost  function  Jg  ti>  the  one  where  the  leading  term  is  now  y|.^,  instead  of  Now,  the 

optimal  control  (*)  is  clearly  linear  in  yt  and  vt-i,  nt  time  t,  and  a  little  algebra  shows  that 
it  can  be  eqpree^  in  the  form  (18a). 

We  now  focus  attention  oh  the  first  inequality  of  (6),  where  agent  B’s  policy  is  fixed  as 
given  by  (16a).  Then  the  problem  faced  by  agent  A  is  one  of  optimal  forecast,  where  the 
cost  function  is  Ja  (given  by  (5a))  and  the  dynamic  constraint  is 

yt+i  ™  (o  +  eSt)yt  +  +  b®t  +  *<+i,  1  <  *  <  T 

“  -(«*/(fc  +  c*)jyT+i  +  «r+2.  t »  r  + 1 

w  (a  +  ecio)yo  +  ^  4’Ci,  t*e0. 

Because  of  the  form  of  the  cost  function,  the  available  Unear-quadratic  theory  cannot  be 
directly  applied  to  this  problem;  nevertheless,  a  one  can  employ  a  iynamie  progrommint  type 


i<f  <r 

t»r  +  l  (18a) 

t  S  0  , 
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argosMBt  to  Goastnict  the  ^timal  solution  in  letrograde  time,  as  in  the  proof  of  Theorem 
2.1  of  Bsfsr  (19ST).  It  has  bm  shorn  in  the  Appmtdix  that  the  (^timal  solution  is  unique 
(under  some  cmditions  which  will  be  spedflad  later),  and  the  optimal  policy  at  time  t  is  a 
function  of  three  variablas,  yt,  %-t  and  «t-i.  The  precise  expression  is 

*  irCv*)  “  s»y*  +  1  <  t  <  r  ,  , 

*  Soito.  t » 0, 

where 


S*  SB  [  ~  6mt)(i54tat  +  (a  +  eS«)mt)  -  6(o  +  eS«)]  (•)’ 

s _ * 

*  h*  +  (1  — 

“  W+(L-hnt)^fi7^  (/»A«i(l  -  +  e0tmt)  -be0t],  (••) 

and  {ntf},  {m<},  {a*}  are  generated  by  (10a)>(10c).  In  writing  down  these  expressions,  we 
have  already  assumed  the  validity  of  (16o)  and  (16«},  since  otherwise  tru  and  fht  would  not 
have  been  well  defined.  We  should  note,  however,  that  even  in  the  pure  forecast  problem 
discussed  in  Ba^ar  (1987),  a  condition  similar  to  (166)  was  required  for  the  well-posedness  of 
the  problem. 

Now,  to  complete  the  derivation,  we  substitute  for  at  and  dt  in  (•)  and  (••)  from  (llo) 
and  (12a),  respectively,  and  observe  that  the  resultmg  expression  for  St  is  identi^  with  that 
of  at,  and  also  when  the  raulting  expression  for  j3t  is  added  to  %,  the  outcome  is  identical 
to  fit'i  in  other  words, 

St  s  Of  ,  0t  +  9=  0t‘ 

When  the  latter  is  used  in  (**)  recursively,  it  follows  that  (vt)  is  generated  by  the  same 
sequence  (of  y^’s)  as  (vt),  and  hence  that  (**)  admits  the  simpler  representation  (17). 

This  then  completes  the  verification  of  the  existence  part  of  the  theorem;  more  precisely, 
of  the  fact  that  the  policies  (17)>(18)  constitute  a  Nash  equilibrium  pair  under  Condition  2. 
Note  that  (I6a)  in  Condition  2  simply  guarantees  that  there  is  a  unique  solution  to  the  two 
pairs  of  coupled  equations  (11)  and  (12),  for  all  t,  and  it  may  also  be  referred  to  as  the  Nash 
condition. 

As  we  have  indicated  earlier,  the  uniqueness  part  of  the  theorem  has  been  verified  sepa¬ 
rately  in  the  Appendix.  o 

Several  observations  and  remarks  would  be  in  order  here.  Firstly,  we  note  that,  as 
opposed  to  the  memoryltM  solution  of  Theonm  1  (obtained  under  myopic  forecast),  the 
unique  Nash  equilibrium  solution  incorporates  memory,  for  both  agents.  For  agent  A,  the 
*best”  forecast  policy  is  a  linear  function  of  the  moot  recent  measurement  and  the  most  recent 
decision  taken  by  that  agent.  (This  is  true  since  it-i  in  (17)  can  be  repiaced  by  0|-i,  without 
affecting  the  solution.]  For  ag^  B,  on  the  other  hand,  the  *best*  tracking  policy  is  a  linear 
function  of  the  most  recent  maasureisent  and  a  linear  aggregate  of  all  past  measuremaats, 
wrighted  in  an  approniate  manner.  The  solution  it  charactarised  in  terms  of  four  gsdn 
coeflicients  {at,St,0t,%),  which  can  be  computed  recursively.  Bence,  the  solution  does  not 
change  structurally  over  time,  which  makes  it  fiaaible  to  obt^  stationary  Nash  policies  for 
the  hifinite-horison  version,  provided  that  the  sequences  {Of'),  (of),  },  (dT)  converge 
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tor  all  fiaita  t  as  T-^oo,  whars  ths  sapsneript  T  ia  ths  ssqasacas  daaotas  ths  dspsndeace  of 
oach  isqnsBcs  on  the  tenainal  time,  t^na  as  a  parameter.  Even  though  the  computation  of 
the  four  critical  (tnaatities  (a<,Si,At4t)  auy  hxA  complicated  at  the  outset,  the  iterations 
are  in  fact  quite  straightfmward,  requiring  rimple  algebraic  maaipulations  at  each  step.  The 
order  one  has  to  follow  in  the  computation  is  as  foilows; 

Starting  at  t^T,  first  compute  the  quadmp/e  (<kr,Sri^,^T)  ^om  (lla)~(12b),  using 
the  given  boundary  conditions  on  Ky-i-u  n*r,  thr  and  nr.  Note  that  this  computation 
involves  the  solution  of  two  pairs  of  coupled  linear  equations,  at  which  point  we  invoke 
the  Nash  condition  (166)  to  obtain  a  unique  solution.  At  this  stage  also  condition  (16c) 
is  invoked,  so  that  (11a)  and  (12a)  are  well  defined.  The  next  step  would  be  to  obtain 
the  new  values  for  kt+t,  rm,  ffh,  nt  at  t=T  + 1,  using  the  iterations  (13a),  (10a),  (106) 
and  (10c),  reapeetivdy.  At  this  stage,  condition  (16a)  is  invoked  so  that  (lOa)  and  (106) 
are  well-deSned  relationships.  These  new  values  for  K,  m,  m,  n  are  then  used  again  in 
(lla)*(126)  to  update  the  values  of  the  gain  coeScients,  and  this  procedure  is  repeated 
until  the  initial  stage  t  =  0  is  reached. 

We  should  point  out  that  similar  to  Condition  1  in  Section  3,  Condition  2  also  generically 
holds,  in  the  sense  that  if  all  but  one  of  the  paramet»  values  are  fixed,  then  there  is  only  a 
finite  number  of  vaiuee  for  that  parameter  for  which  the  condition  fails. 

Even  though  it  is  not  our  intention  to  provide  here  a  general  convergence  analysis  for  the 
infinite-horizon  problem  (this  would  in  fact  be  quite  a  challenging  task),  it  would  nevertheless 
be  instructive  to  study  some  properties  the  stationary  solution,  assuming  that  such  a 
solution  exists  and  Condition  2  holds  for  all  t  of  interest.  Accordingly,  letting 

o*  :=  lim  of,  S*  :=  lim  of,  /?*  :=  Urn  :=  lim  n*  :=  lim  nf, 

T-»ee  T-»oo  T-*«o  T-»ob  T—oo 

it  readily  follows  that  n*s0.  In  view  of  this,  we  arrive  at  the  stationary  Nash  policies 


Vt  =  7*(y*)  =  o*y«  + 

(19a) 

wt  =  M*(y‘)  =  S*yt  + 

(196) 

where  {vt}  is  generated  by 

Vt  =  o*y«-l-/?*v«_i. 

(19c) 

and  the  following  relationship  holds: 

b  ’  ^  6 

(20) 

Now,  using  these  stationary  policies  in  the  system  equation  (4),  we  arrive  at  the  result  that 
the  equilibrium  trajectory  {y^ }  is  generated  by 


V«+i  *  (o  -I-  6o*  +  eS*)y*  -t-  +  ej+i, 

where  {«*}  and  {«( }  denote  the  discrete-time  stochsstk  processes  generated  by  (19a)  and 
(196),  respectively,  when  yt »  yf,  f  >  0.  Note  that,  as  stochastic  processes,  they  are  identical 
almost  swely,  and  hence,  by  alM  using  (20),  it  can  be  shown  that  the  equilibrium  trajectory 
{Vt }  generated  by  the  simpler  dynandcs 


yf+i  »»?_!  + €« 
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which  Admits  the  ARMA  repreaenUtion 


»t+i  +  fi*Vt  -  =  ««+i  + 


(21J 


An  important  observation  that  can  be  made  here  is  that  the  relationship 


Et-ivUi  =  »i-i  . 


holds,  that  is  we  have  perfect  foreeighi.  Said  differently,  the  stationary  Nash  solution  satisfies 
the  side  conation  of  myopic  foresight  introduced  in  section  3,  in  spite  of  the  fact  that  the 
two  solutions  (of  Theorem  1  and  Theorem  3)  are  structurally  different.  [Compare  (20)  (or  its 
stationary  version)  with  (17)-(18).]  This  clearly  implies  that  the  Nash  solution  is  disadvantar 
geoua  to  agent  B  (at  least  in  the  limit  sa  T-»oo),  since  it  does  not  yield  the  best  (optimum) 
solution  obt^able  under  the  side  condition  induced  by  the  equilibrium  solution  itself.  A 
reason  for  this  ineffleietU  behavior  on  the  part  of  B  is  that  in  the  analysis  of  section  3  agent 
A  is  also  an  active  player,  whereas  in  section  2  he  was  passive.  Such  features  can  be  observed 
even  in  finite-horizon  problems,  as  the  following  example  demonstrates. 


Nutnerieat  example  1.  In  our  general  formulation,  let  T=0,  £[po]  and  all  other 

parameter  values  be  unity.  Then,  the  two  solutions  given  in  Theorem  1  and  Theorem  2  and 
the  corresponding  values  of  expected  costs  and  trajectory  sequences  can  be  computed  to  be 
as  follows: 


Theorem  1: 


1  6  1 
t^i  =/ii(Vi)  = -jVi,  Wo  = /io(yo)  = -^yo,  vo  =  ^yo,  (22o) 

f  S  f  S  6 

Ja~  Jb  =  2"^  7*^0. 

2  ,1. 

yi  =  jyo  +  (f.  Vs  =  2^1  + ft- 


Theorem  2: 


1  3  1 

wi  =  ^i(yi)  = -jyii  Wo  =  M5(yo)  = --yo,  wo  =  7o(yo)  = -yoi  (226) 

5  5  15 

•^  =  2-^i6‘^o- 

yj  =  ^yo  +  «i;  yj  =  +  <»• 


A  number  of  observations  can  be  made  in  connection  with  this  example: 

1.  In  both  cases  above,  we  obtain  perfect  forest^  ( i.e.  vo  »  Eoys  ),  but  the  corresponding 
trajectories  are  different.  Even  though  (as  we  have  seen  earlier)  the  Nash  solution  does 
not  generally  enjoy  perfect  foresight  for  the  finito-horison  case,  here  it  does,  mainly  be¬ 
cause  the  problem  involves  basically  a  single  stage,  thus  eliminating  the  effect  of  spillover 
across  consecutive  periods. 

2.  Agent  A  incurs  equal  expected  costs  in  both  cases,  whereas  agent  B  does  worse  with  the 
Nash  solution.  This  is,  course,  consistent  with  our  earlier  comments  just  preceding  this 
example,  which,  even  though  were  made  in  the  context  of  the  infinite-horison  problem, 
are  equally  valid  here  since  the  Nash  solution  satisfies  the  boundary  condition  (i.e.  perfect 
forasight)  of  the  myopic  solution. 
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3.  Since  Ai  =  M*  >>  a  universall]/  optimal  policy  for  acent  B  at  stage  tsl,  whichever 
equilibrium  solution  is  adopted  (even  outside  the  two  considered  here)  the  trajectory 
will  be  given  by 

1 

V2  =  +  ‘2 

Vi  =  Vo  +  Vo  +  Wo- 

Now,  if  we  let  vq  =  £oya>  and  attempt  to  solve  for  vo  from  the  above  eq  '  .ions,  we  first 
obt^  (since  iso  =  Mo(yo)  is  known  to  B  for  each  fixed  po) 

r.  Ir.  Ill 

no  =  ^01/2  =  j-ooVi  =  jVo  +  2*'°  2“’°’ 

&om  which  no  can  be  solved  uniquely  to  give 

Vo  =  Tro(yo)  =  Vo  +  »o:  «vo  =  no(yo)-  (o) 

This  shows  that  the  actual  choice  for  vo  =  'To(yo)  (under  perfect  foresight)  depends 
explicitly  on  B’s  policy  pot  and  the  two  solutions  given  above  are  two  different  manifes¬ 
tations  of  this  dependence.  Both  (22a)  and  (226)  use  (o)  as  a  constraint,  but  while  in 
(22a)  Jb  is  minimized  subject  to  (o),  in  (226)  the  choices  are  determined  by  the  Nash 
solution  of  a  game  played  between  the  two  agents  at  time  r=0.  One  could  envision  other 
scenarios  between  the  two  agents  which  would  lead  to  still  different  choices  for  po  (and 
thereby  'ifo),  but  in  all  cases  the  resulting  expected  cost  to  A  will  be  the  constant  5/4, 
independent  of  po  and  oq.  o 

We  now  conclude  this  section  with  a  second  example,  which  is  an  extended  version  of 
the  previous  example  with  an  additional  stage.  It  will  serve  to  demonstrate  some  addition  nl 
features  of  the  solution  given  in  Theorem  2. 

Hwntrieal  example  t.  In  the  general  formulation,  let  T=:l,  and  all  parameter  values  be  unity. 
Then,  the  unique  Nash  equilibrium  solution  (as  presented  in  Theorem  2)  can  be  computed 
to  be  as  follows: 


1  3 

Wj  =  M;{yj)  =  -■jy2,  v>t  =  Ml(y‘)  =  -|yi  -  0.190476yo. 

*V0  =  #*o(yo)  =  -0.746032yo;  (23) 

vi  -  7r(y‘)  =  -  gVi  +  0.31746yo,  vq  =  75(yo)  =  0.253968yo. 

The  corresponding  equilibrium  tr^ectory  is  generated  by 

yj  =  0.5079366yo  +  «i 
yj  =  0.25vj  +  0.126984yo  +  ej 
yj  =  0.5yJ  +  cs  , 


from  which  it  follows  that  £iy|  ~  0.125y*  +  0.063492yo  ^  7r(yriyo);  that  is,  the  solution 
does  not  lead  to  perfect  foresight  at  time  tsl.  However,  EoVt  ~  0-253968yo  =  7o(yo);  that 
is,  there  is  perfect  foresight  at  (=0.  This  latter  result  is  not  a  feature  of  this  example  only, 
but  holds  for  the  general  solution  of  Theorem  2  (even  though  it  may  be  rather  difficult  to 
prove  algebraically).  Through  an  indirect  reasoning  that  follows  the  proof  of  Theorem  2,  as 
given  in  the  Appendix,  one  can  conclude  that  E0V2  —  7o(yo)  >>  genuine  property  of  the 
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general  Naah  solution  since,  at  the  initial  stage,  the  variable  vo  minimizes  an  expression  that 
is  a  perfect  square  in  vr  (see  and  there  is  no  spillover  effect.  o 

4.  Some  Extensions 

A  first  extension  of  the  results  presented  in  section  2  and  section  3  would  be  to  the  more 
general  case  where  the  reference  trajectory  is  not  zero  and  the  coat  function  (56)  contains 
additional  (time-varying)  weights  on  the  deviation  from  the  desired  trajectory  (t.e.  the  first 
term).  The  reason  why  we  have  not  included  this  in  our  presentation  here  is  because  such 
an  extension  does  not  entail  anything  conceptually  new,  while  requiring  some  additional 
notation  which  would  have  complicated  the  resulting  expressions  considerably.  The  gist  of 
the  results  for  the  nonzero  reference  trajectory  case  is  that  the  statements  of  both  Theorem  1 
and  Theorem  2  remain  essentially  intact,  with  the  only  difference  being  that  now  each  policy 
includes  an  additive  (bias)  term  which  depends  linearly  on  the  desired  reference  trajectory. 
The  existence  conditions  in  both  cases  are  identical  to  the  earlier  ones.  For  the  case  when 
there  is  a  time-varying  weight  in  the  first  term  of  (56),  the  results  again  remain  intact, 
with  only  the  additive  term  1  in  (8a)  replaced  by  this  new  weight  and  Q  in  (13a)  adjusted 
accordingly. 

A  second  extension  would  be  to  the  class  of  problems  where  the  agents  do  not  have 
direct  access  to  the  trajectory  {vt},  but  rather  acquire  common  noisy  measurements  {rt},  as 
defined  by  (16),  where  now  qt  =  z*.  Towards  studying  this  extension,  let  us  assume  that  {ct} 
and  {^t}  sre  sequences  of  independent  Gaussian  zero-mean  random  variables  with  variances 
var((()  =;  ^  >  0,  var((t)  =;  ft  >  0,  and  that  they  are  independent  of  yo  which  is  also  a 
Gaussian  zero-mean  random  variable,  with  variance  oq.  Then,  in  the  formulation  of  section 
2,  we  interpret  the  operator  Et  as  the  conditional  expectation  Note  that  here 

we  have  replaced  rjt  =  z*  with  fj*  :=  (z*,u;*~'),  without  any  loss  of  generality,  since  is 
measurable  with  respect  to  s*~‘.  Now,  letting  i/t  £<yt>  it  is  a  standard  result  (see,  for 
example,  Bertsekaa  (1987)  or  Kumar  and  Varaiya  (1986))  that  yt  is  generated  by  the  Kalman 
filter  equations: 

S/r+r  =  oyr+i  +  cwr+i  +  [^t+2/(^t+3  +  fr+j))'’r+2,  , 

(24ci] 

y«+i  =  oy*  +  6Kfy«+i  +  ctot  -I-  \at+il(ot+i  +  f«+i))rt+i,  t  ST-,  y_i  =  0, 

U+i  •■=  z*+j  -  ay,  -  bEtyt+2  -  cm,  (246) 

^f+i  =  +  m+i,  ^0  =  (24e) 

where  {r,}  is  a  sequence  of  independent  Gaussian  random  variables,  known  as  the  innovotion 
sequence.  In  writing  down  these  relationships,  we  have  made  explicit  use  of  the  fact  that 
both  Etyi+2  and  m  nre  «‘-meaaurable. 

Now  note  that  the  error  sequence  {«,},  e,  :=  y,  —  y,,  is  generated  by 

Cf+i  =  ae,  +  tt+i  —  «o  =  0,  (A) 

and  that  =  0  for  all  n  >  1.  In  view  of  this  last  property, 

■E«y«+a  =  Etyt+2  +  Btct+2  =  Etyt+2, 
and  hence  (24a)  can  be  rewritten  as 

iT+2  =  “yr+i  +  cwT+i  +  !^r+j/(^r+j  +  fr+j)l'’r+j, 

i/«+i  =  «y«  +  +  eu't  +  lff«+i/(®»+i  +  ft+i)l»’t+i,  t  <  T;  y-i=0. 


(25) 
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fWthenDOM,  aiace  (h  »  ^  +  ct,  and  ih  it  (vthogonal  to  «,  th«  coontorpart  of  (7a)  for  tha 
noisy  case  would  be: 

T+l  T+l 

+1  =  £  EiPUi  +  WH’*  +  E  (26) 

tm»  tmt 

where  the  second  summation  term  does  not  enter  the  optimisation,  since  the  sequence  {st} 
generated  by  (A)  is  independoit  of  the  control  sequence  {wt}.  Hence,  the  problon  faced  by 
agent  B  is  the  minimisation  of  the  first  term  of  (26)  subject  to  the  dynamics  (25),  where 
wt  —  which  is  compatible  with  the  original  information  =  (y*,w*~^)  since  pt  is 

generated  by  (qt-iiWt-i)*  Then,  the  problem  is  identical  with  the  perfect  information  case 
(apart  from  a  change  of  notation),  in  view  of  the  fact  that  {rt}  is  a  sero-mean  independent 
sequence,  playing  the  role  of  {<«}  in  (7b).  This  shows  that  the  problem  (with  myopic  forecast) 
features  certainty  equivalence,  malcing  the  statement  of  Tbeoram  1  valid  also  in  the  noisy  case, 
with  only  yt  replaced  by  yt,  and  (9e)  including  an  additional  positive  term  due  to  the  second 
term  of  (26).  The  following  theorem  summarizes  this  result. 

Theorem  3.  Let  Conditioa  1  be  aatisSed.  Then,  the  dynamic  policy  optimization  problem 
with  myopic  forecast,  as  formulated  in  section  2  but  with  common  noisy  measurements  (lb) 
for  both  agents,  admits  the  unique  solution 

uit  =  A«(y»)  =  ytSt,  0  <  t  <  T  + 1,  (27a) 

with  the  corresponding  forecast  policy  given  by 

(271) 

Vt+lUt 

where  {$<}  is  generated  by  (25),  and  {gt},  {</<}  are  as  defined  by  (8e)  and  (8b),  respectively. 


Hence,  for  the  noisy  case,  certainty  equivalence  bolds  under  myopic  forecast,  and  the 
statement  of  Theorem  1  basically  remvns  intact.  For  Theorem  2,  however,  there  is  no  direct 
counterpart,  and  derivation  of  the  Nash  equilibrium  solution  is  quite  a  nontrivial  task.  We 
will  not  pursue  this  extension  here,  since  presenting  the  full  details  of  the  derivation  of  the 
Nash  equilibrium  solution  would  at  least  double  the  length  of  the  present  paper.  What  we 
can  say  at  this  point,  however,  is  that  (guided  by  the  results  presented  in  Baqar  (1978b)  for  a 
linear>quadratic  nontero-sum  dynamic  game  with  a  different  type  of  an  information  pattern 
and  a  different  type  of  a  cost  function  for  one  of  the  agents)  the  problem  will  generically 
admit  a  unique  Nash  equilibrium  solution,  linear  in  the  available  common  information.  This 
solution  will  not  satisfy  the  certainty  equivalence  or  separation  principle  of  stochastic  control, 
and  thus  will  have  no  relationship  with  the  solution  presented  in  Theorem  2.  The  following 
numerical  example  (which  is  the  '^oisy*  version  of  the  second  example  of  section  3)  should 
serve  to  corroborate  this  claim  and  to  give  some  indication  as  to  the  intricacies  involved  in 
the  derivation  of  the  general  solution. 

Numerical  example  S.  Consider  the  second  numerical  example  of  section  S,  but  with  noisy 
measurement  (lb)  for  both  agents,  and  with  all  parameter  values  (including  the  noise  vari¬ 
ances)  equal  to  unity.  Hence,  the  cost  functions  are 

Ja  =  E{(vt  -  ys)*  +  (uo  -  V»)*> 

Jb  =  E{yi  +  tej  +  y|  +  wf  +  Vi  +  Wo}> 


(28) 
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and  the  dynamic  constrainta  are 

US  =  to  +  toj  +  <8 

to  “  Vi  +  *1  +  (29) 

to  =  to  +  *0  +  Wo  +  «o. 

where  wj  =  *ia(**),  Wi  =  *“0  =  Mo(*o).  Oi  =  '7i(*‘)i  »o  =  loCao);  *o  =  Vo  +  to, 

to  =  Vi  +  and  «2  =  to  +  to-  The  first  significant  difference  between  the  perfect  and 
the  noisy  measurement  cases  appears  in  the  construction  of  the  best  H2,  which  now  depends 
explicitly  on  (miiM)  end  (^iiiro)*  [Recall  that  in  the  perfect  measurement  case  covered  by 
Theorem  2,  there  was  a  universally  optimal  policy  for  agent  B  at  the  terminal  stage  of  the 
game.]  With  the  quadruple  ('Ti.iroiPitMo)  fixed,  say  at  the  minimization  of  Jb  with 
respect  to  ^2  becomes  a  standard  quadratic  optimization  problem, 

min£{(y2  +u>7  +  <a)*  + 


whose  unique  solution  is 


W2  =  M2(a*,Tf,/i)  =  -~E[y2\z^,'i,ti\  =:  ^to-j^- 


(30) 


Here  vs-tm  w  generated  by  the  Kalman  filter: 

g 

=  OiioMo  +  'ri(**)  +  Mi(*‘)  +  jg(to  -  finoMo  -  'rx(*‘)  -  Mi(*')) 

3 

toioMo  =  to  +  Tfo(*o)  +  #*o(to))  +  g(to  -  to  -  'iro(A))  -  Mo(zo))  (31) 

to  =  \zo, 

which  depends  on  (-7,^)  partly  directly  and  partly  through  :=  ff[yils‘,7o,to>l>  To 

obtain  the  pair  that  is  in  Nash  equilibrium  with  (30),  we  follow  a  procedure  quite 

anaiogous  (in  principle)  to  the  one  followed  in  the  proof  of  uniqueness  (for  Theorem  2)  in 
the  Appendix,  geared  towards  obtmning  a  (unique)  stagewise  equilibrium.  Accordingly,  the 
derivation  involves  the  solution  of  two  static  games,  one  at  t=l  and  the  other  one  at  t=0.  To 
characterize  the  static  game  at  t=l,  we  substitute  (31)  into  (28),  eliminate  the  intermediate 
variables  and  take  expectation  over  the  statistics  of  cs,  <2  and  (2,  to  arrive  at  the  reduced 
conditional  (on  z^)  cost  functions: 

“  *(v»  +  +  fto  +  j(7i  +  Ml)]* 

+  (to-Vi  -to  -wi|*|z*}, 

+  to  +  wi)  -  |to  -  |(7i  +  /ii)]® 

+  +  71  +  Ml)  +  8(yi  +  Vi  +  Wi)|*  +  (yi  +  »i  +  wi)*  +  w*  |s‘}. 

In  the  above,  we  have  made  notational  simplifications  by  suppressing  the  (7o<  Mo)*dependcnce 
of  f/i  and  the  arguments  of  (71,^1).  This  is  clearly  a  static  game  in  the  pair  (vi,  wi),  and  its 
Nash  solution  can  be  obtained  for  each  fixed  (71, Mi)  and  (7o,Mo)i  where  we  take  oo  =  7o(s«)). 


Diffaraatlating  with  roqMct  to  «i  and  with  napoct  to  wi,  and  aatting  tha  raaulting 
fiTprwaifMia  aqnal  to  aero  aft«  eonditioniag  on  a*,  wa  flad  that  the  Naah  conditkm  is  aatisfiad 
and  there  axiata  a  aai<{ae  aolution  to  the  pair  of  ecptationa,  linear  in  'll,  Hi,  >nd  no-  Now 
requiring  conaistency  in  the  aolution  (aa  in  the  proof  of  uniqueneaa  for  Theorem  3  in  the 
Appendix),  we  aet  vi  s  *yi(s*),  toi  iti{*^),  and  solve  the  reeulting  pair  of  linear  equations 
(in  vi  and  wi)  uniquely,  to  arrive  at  the  policies: 

t>i  =  '»i(*‘.TO.Po)  =  -0A23810fix  +  1.54761fl^o(*o),  {32o) 

Wi  ®  Pi(**»TOtPo)  =  ~0*285714^i  —  0.9MS71'7o(eb)-  (324) 

Note  that  here  ‘yo  is  yet  to  be  determined. 

To  complete  the  solution,  we  next  formulate  the  game  at  t=0,  by  substituting  (30)  and 
(32)  into  (28),  again  eliminating  the  intermediate  variables  and  averaging  over  the  statistics 
of  ^e  random  variables  involved,  to  obtain  the  reduced  conditional  (on  so)  cost  functions: 

=  f;{(0.619048«o  +  1.208790yo  -  0.029304po  +  0.648352(vo  +  wo))* 

+  (0.48S714VO  +  0.29S238yo  -  0.514286ti>o  +  0.1fl0477yo  +  0.809S24po)’|«o}, 


Ji  =  je{7(0.619048yo  -  1.194139vi  +  1.38461Svi)* 

4 

+  j(0.619048yo  -  0.424909yi  +  0.615385yi)*  +  (0.619048yo  -  0.809524yi  +  yi)* 

+  (0.28S714yi  +  0.928S71yo)*  +  »?  +  wS|so}, 

where  both  yi  and  Hi  depend  on  (vo.wo),  the  latter  through  si,  as  given  in  (31). 

The  procedure  here  is  the  same  as  at  tsl:  First  obtain  the  Nash  solution  of 
in  terms  of  ('jotHo),  then  require  consistency  (vos7o(<o)>wo=po(so))  and  solve  for  (vo,ufo) 
from  the  resulting  equations,  which  will  lead  to  policies  whose  argument  is  zq.  At  each  step 
the  uniqueness  condition  is  met,  and  thus  the  procedure  yields  the  unique  Nash  equilibrium 
policies  (at  (=0); 

VO  =  yJ(*o)  =  +0  J482275o,  (33o) 

wo  =  /to(^)  =  -0.751773jJo.  (334) 

These  policies  are  finally  used  in  (32)  and  (31)  to  complete  the  characterization  of  the  Nash 
equilibrium  policies: 


where 


V,  =  =  -0.523810y?  +  0.384161fo. 

(34o) 

wi  =  pT(*‘)  =  -0.288714J?  -  0.230496JO, 

(344) 

wj  =  #*5(**)  =  -0.5  j?j. 

(34c) 

y?  s  0.050102$o  +  0.073260$?  +  0.615385x2 

$?  s  0.198582yo  +  0.6si 

(35) 

$0  =  0.5so. 

An  equivalent  representation  for  rti  in  (34a)  would  be 


»i  s  y*(s‘)  =  -0.523810(1?  +  1.547619V0, 
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which  shows  explicit  dependence  on  oo-  Note  that  the  policies  (34)  are  different  from  their 
counterparts  in  the  noise-free  case  (i.e.  (23)),  thus  corroborating  our  earlier  remark  that  the 
*noisy  version”  does  not  feature  certainty  equivalence. 

The  equilibrium  trajectory  corresponding  to  the  unique  Nash  solution  is  generated  by 


yj  =  yj  -  0.5yJ  +  «3 

0.809524^  -I-  O.1S36640q  -I-  ej 
yj  =  yo  ~  0.503546^  +  ti. 


Using  these,  it  is  easy  to  check  that,  as  in  the  second  example  of  section  3,  Eij/g  ^  7i(s*), 
while  EovX  —  'io(‘%)>  which  shows  that  the  Nash  solution  could  lead  to  perfect  foresight  at 
the  initial  stage,  even  in  the  noisy  case.  As  we  will  discuss  in  a  companion  paper,  this  turns 
out  to  be  a  general  property  of  the  Nash  solution  for  the  “noisy  version”  of  the  problem  of 
section  S.  o 

Appendix 

In  this  appendix,  we  first  complete  the  proof  of  the  existence  part  of  Theorem  2  by 
showing  that  the  policy  (**)  given  there  indeed  solves  agent  A’s  optimization  problem.  Sub¬ 
sequently,  we  establish  the  uniqueness  of  the  Nash  solution  presented  in  Theorem  2. 
Existence.  The  optimization  problem  faced  by  agent  A  is  the  minimization  of  Jq*,  where 

tmt 


under  the  constraints 

yr+j  =  (afc/(*  +-e*)]yT+i  +  er+j 
Vt+i  =  («  +  +  e0tvt-i  +  +  «t+i,  1  <  t  <  T 

yi  =  (o-l-cSo)yo  +  «i; 
vt  =  a«y»  +  0tvt-u  v«  =  7*(y‘)' 

We  now  claim  that,  for  a  general  t, 

dm  =  min  £{pAn«+i(v«+i  -  mt+iyt+2  —  m»+iVf+i)* -I- (vt  -  yt+s)*}  +  qt,  (A.l) 

where  {ft}  is  a  sequence  depending  only  on  the  variances  of  the  additive  stochastic  terms  et, 
t  <  T  +  2.  Under  the  validity  of  this  assertion,  the  optimal  policy  at  time  (  is  obtained  by 
minimizing  the  following  quantity  with  respect  to  the  scalar  variable  vt.fi=;v,  for  each  fixed 

yt+t. 


~  fnt+i(a  +  c3t+i)yi+i  —  mt+icfit+ivt  —  mt+iiv  —  mt.).iat.t.iyt.t.i 

-  mi+i0t+ivtl^  +  l(o  +  eSt+t)yt+i  +  e0t+ivt  +  hu  -  »tl*|y*'^‘}. 

Being  quadratic  and  strictly  convex  (in  «),  this  optimization  problem  admits  a  unique  solution 
(for  each  fixed  y«+i,«t,o<),  given  by 

»t+i  =*  7«+i(y*'*'*)  =  &«.|.iy(  +  A+i*i  +  0t+iVt,  M-3«) 
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fbrO<(<T  —  1,  and  at  the  initial  stage  by 

»o  »  To(l«>)  *  dolto,  (A.36) 


where 

^  +  (1  -  ^**^***^^  ~  +  (*  +  ««*)"»«)  -  Ktt  +  eSt)l 

g  _ ft _ 

*  6*  +  (l  — 

"  6a  +  (1  -  +  eftmt)  -  befit]. 

Aa  we  have  discussed  earlier  (in  the  proof  of  the  existence  part  of  Theorem  2),  substitution 
for  a«  and  fit  (from  (11a)  and  (12a),  respectively)  into  the  three  expressions  above,  leads  to 
the  equivalences  dit  =  and  ^t  fit  ^  fit.  Hence,  the  optimal  solution  (A.3)  admits  the 
equivalent  representation 


vt  =  atVt  +  fiit>t-i  +  {fit  -  fit)n-i,  i<t<r 

=»aoyo  ,  t  =  0. 

We  now  turn  to  verification  of  the  structural  form  (A.1).  The  result  trivially  holds  for 
t=T,  with  mr  =  <tk/{k  +  ea),mr  &=  0.  Let  us  therefore  assume  the  validity  of  the  assertion 
for  t+1  and  prove  it  for  t.  Towards  this  end,  we  substitute  (A.4),  with  t  replaced  by  t+1, 
into  (A.2),  and  arrive  (after  some  rather  tedious  algebra)  at  aa  expression  which  is  a  perfect 
square  in  ot>  Vt+i  and  St: 


E{nt{vt  -  mtyi+i  -  mt»t)*ly‘'''‘}.  (A.5) 

Here  mt,  mt  and  nt  are  defined  in  terms  of  mt^i,  mt+i  and  nt-fi  as  in  (10a)  through  (lOe).  [In 
fact,  it  is  not  difficult  to  see  that  the  resulting  cost  should  be  a  perfect  square,  because  (A.2] 
can  be  made  equal  to  zero  by  appropriately  choosing  vt  and  vt+i.  With  this  observation,  it 
then  remains  to  find  the  three  coefficients  tit,  mt  and  mf]  Now,  since  the  minimiiTn  of  (A.2) 
over  vt.t-1  is  equal  to  (A.S),  we  have 

min  Jt-i“  “in  E{{vt-i  -  Vt+i)^  +  Pa  ,  “in  Jf) 

=  min  .H{(»t-i  -  Vi+i)*Patii(wt  -  mtpt+i  -  mtui)*) 

+  +  (1  +  ^a»»t+i“t+i)*'**"(*«+j)l» 

which  is  in  the  same  form  as  (A.l),  with 

qt-i  :*  PA\qt  +  (1  +  Pa»«+i»»*t+i)»«’(««+z)l- 

This  then  completes  the  proof  of  optimality  of  (**)  in  the  proof  of  the  existence  part  of 
Theorem  2. 

Uniqueness.  It  is  a  weU>known  fact  that  dynamic  games  could  admit  nonunique  Nash  equilib> 
ria,  with  each  such  equilibrium  leading  to  a  different  cost  pair  which  are  in  general  incompara¬ 
ble  (see,  for  example,  Hager  and  Olsder  (1982)).  Thus,  *uniqaaaass”  is  an  important  quaatkm 
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to  poM,  if  the  propoeed  equilibriiun  is  to  be  of  vahie.  As  we  have  discussed  extensively  in 
earlier  papas  (for  exaoqtie,  B»tiu(10r6),  Baf»r(i077)),  the  main  source  of  noauniquensss  in 
Nash  equiUhria  te  the  ao<aIled  in/ormatioM/  nommtfusnass  erhich  arises  if  each  afsnt,  in 
his  <me-dded  opthaiaation,  has  the  freedom  of  choosing  different  rspresegtations  of  the  same 
policy.  What  we  prove  in  the  sequel  is  that  fw  the  ganae  problem  covered  by  Theorem  3 
there  is  no  informational  nonuniqueneu,  and  the  structural  form  (17)-(18)  is  ihe  only  form 
in  which  a  Nash  eqniUbrium  can  exist.  Furthermore,  we  show  that  structural  uniqueness  is 
guaranteed  under  Condition  F.  In  the  proof,  we  will  not  o^lieitly  derive  the  expresskms  for 
this  unique  Nash  solution,  since  we  have  already  shown  in  the  first  part  of  the  pnxff  that 
(17)>(18)  exists  aa  a  Nash  eqmlibrium. 

Tosrards  devising  a  proof  for  uniqueness,  we  first  introduce  two  gtnerie  functions  quad(*) 
and  lia(')>  where 

qttad(‘)  =  a  quadratic  foiKtlon  of  its  arguments 
lin(')  =  a  linear  function  of  its  arguments. 

Furthermore,  we  introduce  a  class  of  netted  tubgomet  {G*},  parameterized  by  s,  each  one 
being  a  replica  of  the  original  game  but  defined  on  a  shorter  time  interval,  [s,  T  + 1],  0  <  s  < 
T  + 1.  More  precisely,  for  the  subgame  G«,  the  coet  functions  are  defined  by  (5a)-(5h)  with 
the  lower  limits  changed  to  t=s—l,  and  with  the  action  variables  bring  ;=  (tfr>"«v«4.i,Oj) 
for  A,  and  :=  for  B,  where  vt  =  M<(y‘)i  tot  =  and  a  similar 

convention  as  above  applying  to  the  policy  variables  To  be  consistent  with  this 

convention,  for  s=0  we  extend  the  limit  of  the  summation  to  t  =  — 1  in  both  Jx  and  Jb, 
by  adding  tero  as  the  incremental  coat  term  at  t  =  —1.  Now  let  (7  :=  := 

a  Nash  equilibrium  solution  for  the  ori^al  game  (Go),  such  as  the  one  given  in  Theorem 
2.  Then,  it  is  a  well-known  property  of  the  Nash  solution  (called  the  stagewiee  eguilibrium 
property)  that  for  any  s,  the  truncated  version  of  these  policies,  constitutes  a 

Nash  equilibrium  solution  for  G«,  with  the  past  policies  )  fixed  at 

We  now  develop  a  procedure  for  studying  the  uniqueness  ot  the  solutions  of  these  indi¬ 
vidual  subgames.  First  consida  the  case  s  »  T  -I- 1,  where  Gr^-i  is  not  really  a  game  but 
a  ono-sided  optimisation  problem  for  agent  B,  since  only  B  is  active  at  t  s  T  +  1.  Then, 
clearly  the  solution  is  unique,  and  is  given  by  the  second  line  in  (18a).  Note  that  this  solution 
is  both  informationally  and  structurally  unique  (regardless  of  the  past  policy  choica),  the 
forma  being  due  to  our  assumption  in  section  1  on  the  structure  of  the  probability  distribu¬ 
tion  of  the  additive  system  noise.  Hence,  in  the  study  of  the  second  game  in  the  sequence, 
Gr,  we  can  take  itr+i  aa  in  (18a),  without  any  loss  of  generality.  Accordingly,  substituting 
this  HT+it  say  ftr+i,  into  both  7x  uu^  <f0>  eliminating  the  Intermediate  variables  using  the 
evolution  equation  (4)  and  avaaging  ova  the  statistics  of  the  random  variables  by  employing 
their  indejiendence  property,  we  arrive  at  the  structtual  forms 

co8fx(GT)  *  qua<f(yr,WT. wr.WT-i) 
costn(Gr)  =  qusd(yr,vr.  wr)  +  quad(w|>_i}, 

which  are  the  costs  incurred  to  A  and  B,  respectively,  conditioned  on  the  information  available 
at  time  T,  i.s.  qysy^.  Since  the  first  cost  shows  explicit  dependence  on  vy-i,  we  fix 
vt-1  snrr-i(y^~*),  and  solve  for  the  Nash  equilibrium  M  the  rsnulting  static  game.  Because 
of  the  quadratic  structure  of  the  cost  functions,  the  Nash  solution,  if  it  exists,  will  be  linea 
in  the  pair  (yriVr-i);  furthermore  it  will  be  (structurally)  unique  unda  conditioiu  not 
depending  on  yr  and  «r-ii  ufi  Condition  2  precisely  servw  this  purpose.  Hence,  the  static 
game  d^ned  ^  (A.6)  admits  a  unique  Nash  solutioa,  for  each  fixed  7r-ii  given  by 

VT  =  7T(y*')  a  ffa(yT.  »T-i)  (A.7a) 
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* 

)aMyT.OT-i).  (X.76) 

when  vr-i  »  Tlu  linear  functions  here  are  precisely  the  ones  ghnen  m  (17) 

and  (18),  with  vy-t  in  the  latter  case  replaced  hy  ?r.|.  The  solution  is  also  unique  rep* 
rts*ntatiommM  since,  because  of  our  nonsinfular  statistics  assumption  on  the  probability 
distributions  of  the  random  variaUes  involved,  yy  cannot  be  expressed  in  tenns  the  past 
values  of  the  trajectory  tdmotl  surely  (which  would  have  been  possible  in  a  purely  determin* 
iatk  problem).  We  now  note  that  the  conq^lete  (unique)  solution  to  subgame  Gy  is  (X.7) 
akatg  with  which  was  the  unique  aoluUon  (te  agent  B)  to  subgame  Gt+x- 

The  next  game  in  the  sequence,  Gy.i,  involves  the  the  action  variables  (vy,«y_t)  for 
agent  A  and  (tvy4.i,toy,toy-i)  for  agent  B.  Since  every  Nash  equilibrium  is  necessarily  a 
stagewise  equilibrium  and  since  the  uni<pie  (linear)  Nash  solution  of  Gy  does  not  depend 
structurally  on  vy-i  and  wy.i,  it  follows  that  every  Nash  equilibrium  for  Gy_i  should 
match  with  that  of  Gy  far  policies  UT+tt  sad  ‘yy.  Hence,  the  equilibrium  solution  of 
Gy-i  will  be  nonunique  only  if  the  last  components  of  the  policy  sequences,  (7y-i,/«y-i), 
are  nonunique  at  equilibrium.  Towards  a  study  of  this,  we  substitute  the  solution  of  Gy  into 
Jjx  and  Ja,  with  «y-i  in  (A.76)  replaced  by  a  general  function  of  y^~*,  say  ^-i(y^~‘), 
since  B  does  not  have  direct  access  to  vy-i.  (It  is  important  to  note  at  this  point  that  if 
B  had  direct  access  to  t>y-i,  the  solution  would  have  been  informatiomdly  nonunique,  for 
reasons  discussed  extensively  in  Ba^ar  (1978a)  for  a  different  class  of  such  games.]  Now,  after 
eliminating  the  intermediate  variables  and  avera^ng  over  the  stochastic  variables,  we  arrive 
at  the  following  reduced  costs  for  Gy-i,  conditioned  on  the  common  information  available  at 
time  r— 1,  i.e.  y*""*: 

coetx(Gy_i)  *  quad(yy-i,»y-i,wy_i,»y_8,^fry-i(y*'“^)) 
costB(Gy_i)  =  quad(yy_i,vy-i,wy-i,dv-i(y’'"‘)). 

Here,  in  addition  to  the  unknown  (but  fixed)  function  ^y.i,  we  also  have  vy-j  s  7y-a(y^~’) 
fixed  by  an  arbitrary  choice  of  iry-a.  Under  an  appropriate  condition  which  is  independent  of 
th'-i  and  7y.a  (which  is  also  guaranteed  by  Condition  2),  this  static  game  admits  a  unique 
equilibrium  for  each  fixed  5^y-i  and  7y-a: 

VT-i  =  '?T-i(vy-i.»T-a,tfry-i(yy-i))  s  iin(yy_i,wy_a,0y_j(yy_i))  (A.8a) 

t»y-i  =  ilr-i(yy-i,vy-j,d^-i(yy-i))  s  lin(yy_i,»y_a,i^-i(yy-i)),  (A.86) 

where  «y-a»7r-a(y^~’)<  Next,  we  impose  consistency  in  the  solution  for  each  fixed  7y-a, 
which  requires  that  ^y-i  s  ^y-i.  Using  this  side  conation  in  (A.8a),  we  arrive  at 

vy_i  =  lin(vy_i,»y_a,vr-i) 

which,  being  lir.''!'  admits  the  unique  solution  (for  each  fixed  y^~‘  and  7y-a) 

VT-i  *  lr-i(yy-i,vy-a)  ■  Mw-j.^r-a).  (A»«) 

under  a  nonsingularity  condition  which  is  met  under  Condition  2.  Letting  d^y-i  »  7r-i  in 
(AM),  we  finally  obt^  for  wy-i  (for  each  fixed  'tr-t)- 


VT-i  =  ?T-i(yy-i,vr-a)  »  Mvr-i.vr-a)  (-d-Sh) 

This  then  completes  the  verification  of  the  uniqueness  of  the  solution  of  Gy-i,  for  each  fixed 
7y-a.  Note  t^t  the  complete  solution  to  Gy-i  is  given  by  My+i>  (^d.?)  and  (A.O),  with 
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vr-i  in  {A.7b)  r^laced  by  th«  «xpr«nion  in  {ii.9a).  H«re  w«  could  alio  have  ejqtreaaed 
(i4.7a)  in  tenns  cS  y^,  instead  of  (yr<«r-i}>  hy  luhstituting  for  vy-i  from  (A.9a),  but  this 
is  not  necessary  since  agent  A  have  access  to  his  past  decision  value,  and  enriching 
lus  infocihatton  set  by  also  including  past  deciston  values  does  not  lead  to  informational 
nonuniqueness. 

The  important  observation  here  is  that,  for  each  fixed  the  solution  of  subgame 
Gr-i  (to  be  denoted  (7r,7r-iiMr-fi>i<r,Mr-i) )  >■  atmcturally  unique,  with  each  strategy 
being  linear  in  its  arguments.  More  precisely,  we  have  linear  in  (pr,vr>i),  ^-i  linw 
la  (yr-i,»T-*)t  ?T+i  fo»«»r  in  yr+u  5t  linear  in  (vT,»r-i,'»r-a(y’’"*))  •bA  Mr-i  linear 
in  (yr-i,7r-2(y^*'’))*  Furthermore,  the  solution  is  informationally  unique  because  of  the 
nonsingular  statistics  of  the  additive  noiM  in  the  dynamics  (4).  Then,  in  the  construction 
of  the  Nash  solution  for  subgame  Gt-s,  we  first  substitute  for  ('yr,'Tr-i;Mr-«-t,/tr,Mr-i) 
from  the  unique  solution  of  Gr-i,  with  'it-3  replaced  by  a  general  function  i>T-2,  as  in  the 
construction  of  the  solution  for  Gr-i.  Repeating  the  same  procedure  as  in  G]>-i,  we  can 
obt^  a  linear  stagewise  Nash  solution  for  Gt~2  for  each  fixed  'rr-s,  whose  uniqueness  is 
again  guaranteed  by  Condition  2.  Following  this  procedure  in  retrograde  time,  we  find  that 
for  each  e,  the  subgame  G,  admits  a  unique  stagewise  equilibrium  (for  each  fixed  linear 
in  the  available  information  as  well  as  in  7«-i.  Since  7-1  is  trivially  sero,  the  process  halts 
at  leading  to  the  conclusion  that  the  game  Go  admits  a  unique  stagewise  equilibrium, 
linear  in  the  common  information  available  to  the  agents.  This  then  establishes  uniqueness 
of  the  Nash  solution  of  the  original  problem  (which  is  identical  with  Go),  since  every  Nash 
equilibrium  is  a  stagewise  equilibrium  and  we  have  already  proven  that  the  game  admits  at 
least  one  Nash  equilibrium. 

We  conclude  this  Appendix  by  pointing  to  the  fact  that  the  above  procedure  would  have 
been  an  alternative  method  for  the  construction  of  the  Nash  solution  given  in  Theorem  2,  but 
alone  it  would  not  be  stifficient,  since  a  stagewise  equilibrium  need  not  be  a  Nash  equilibrium, 
o 
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Abstract  This  paper  deals  with  the  construction  of  cooperative  equilibria  for  stochastic  dynamic 
games,  where  the  players  cannot  observe  the  actions  of  their  opponents.  For  a  particular  class  of 
dynamic  games  with  payoffs  defined  as  the  limit  of  average  gains  one  establishes  the  existence  of 
perfect  equibria  which  are  also  Pareto-optimal. 


1.  Introduction 

The  aim  of  this  paper  is  to  explore  the  possibility  to  construct  cooperative  equilibria  in  stochastic 
sequential  games  of  infinite  duration  with  payoffs  defined  as  linut  of  averages.  A  stochastic  sequential 
game  is  a  discrete-time  dynamic  game  that  involves  an  element  of  uncertainty  represented  by  a  random 
noise  affecting  the  state  transitions.  Sequential  games  include  as  a  particular  case  the  class  of  so-called 
repeated  games,  which  arise  when  a  static  game  (e.g.  a  matrix  game  or  a  Cournot  duopoly  game)  is 
played  repeatedly  over  an  infinite  number  of  periods. 

One  of  the  most  interesting  features  of  dynamic  game  theory  is  that  it  allows  the  study  of  cooperative 
or  collusive  behavior  among  the  agents  engaged  in  the  control  of  a  dynamic  system,  even  in  the  absence 
of  any  external  mechanism  which  makes  cooperative  agreements  binding.  Recall  that  the  presence  of 
such  a  "cheating  preventing”  device  is  a  precondition  of  cooperation  in  static  games  (Luce  and  Raiffa 
1957).  Cooperative  solutions  of  dynamic  games,  on  the  other  hand,  can  be  supported  by  "cheating- 
proor  equilibrium  strategies,  which  imply  that  a  player  react  with  a  punitive  action  to  any  breach 
of  cooperation  by  his  partners.  This  fact  has  been  first  established  for  deterministic  repeated  games 
(see  Aumann  1959,  Friedman  1977,  Rubinstein  1979,  1980,  and  Radner  1980),  and  later  studied  in 
the  more  general  context  of  deterministic  sequential  and  differential  games  in  Tolwinski  1982,  1986, 
Hanrie  smd  Tolwinski  1984,  1985,  and  Tolwinski  et  al.  1986.  The  collusive  equilibria  of  dynamic 
games  considered  in  these  works  have  been  obtained  under  the  assumption  that  a  player  making  his 
decision  at  a  given  instant  of  time  has  complete  information  about  his  partners’  action  history.  The 
importance  of  this  assumption  stems  from  the  fact  that  it  ensures  that  each  player  has  the  ability  to 
detect  any  breach  of  cooperation  by  other  players,  and  then  to  react  to  it  in  an  appropriate  manner. 
The  question  addressed  in  the  present  paper  Is  whether  the  above  assumption  can  be  relaxed,  i.e.,  can 
a  cooperative  equilibrium  be  constructed  for  a  dynamic  game,  where  the  players  have  only  incomplete 
information  abont  other  players  actions? 


/ 


The  iHM  of  exiitettce  of  cooperative  eqnilibrie  in  repented  gnmee  when  the  informntion  held  by  ench 
plnyer  nboat  hit  opponents  notions  is  distorted  by  mndom  noise  hns  been  nddressed  by  Rndner  1981  or 
Rubinstein  nnd  Ynnri  1983  for  the  cnse  of  nvemge  pnyoff  criterin,  nnd  Rndner  1985  ,  Porter  1983,  Green 
and  Porter  1984  and  Fudenberg  and  Maskin  1986,  among  others  for  the  case  of  discounted  payoffs. 
Rndner  considered  the  problem  of  monitoring  cooperative  agreements  in  the  context  of  a  repeated 
principal-agent  game,  and  obtained  cooperative  epsilon  equilibria  under  the  assumption  that  the 
players  maximize  thrir  average  payoffs  over  a  Unite  but  arbitrarily  large  number  of  periods.  Radner’s 
approach  has  been  closely  related  to  the  idea  of  sequential  tests  of  power  one  (Robbins  and  Siegmund 
1974);  it  takes  advantage  of  the  fact  that  only  the  changes  of  policy  that  are  maintained  for  relatively 
long  periods  of  time  can  have  any  noticeable  impact  on  long-term  average  payoffs.  The  changes  of  this 
type,  on  the  other  hand,  can  be  detected  by  means  of  statistical  tests.  Radner’s  approach  involves 
a  so-called  triggering  mechanisms;  a  strategy  is  then  a  combination  of  a  cooperative  policy,  a  threat 
(or  punitive)  policy,  and  a  switching  rule  which  triggers  punitive  retaliations.  The  repeated  game 
structure  is  not  essential  for  the  obtention  of  such  cooperative  equilibria  as  it  will  be  shown  in  the  rest 
of  this  paper  which  establishes  a  result  similar  to  Radner’s  in  the  realm  of  sequential  stochastic  games. 
However  an  inffnite  horizon  setting  is  essential  for  such  equilibria  to  exist,  since  as  shown  by  Basar  1977 
a  stochastic  sequential  game  played  over  a  Unite  time  horizon,  contrarily  to  deterministic  sequential 
games  typically  admits  as  equilibria  only  those  which  correspond  to  the  strictly  noncooperative  mode 
of  play  (viz.  the  feedback  Nash  equilibria  obtained  through  the  dynamic  programming  approach). 
What  makes  the  situation  different  for  infinite  horizon  stochastic  sequential  games  is  the  fact  that 
there  is  always  enough  time  for  a  player  to  retaliate  if  cheating  has  been  detected.  When  the  strategy 
evaluation  criteria  defining  the  players  payoffs  are  the  limits  of  the  average  transition  rewards,  then 
only  the  long  term  effects  of  strategic  choices  really  matter.  This  fact  also  facilitates  the  construction 
of  efficient  cooperative  equilibria.  . 

The  paper  is  organized  as  follows.  In  Section  2  the  definition  of  the  stochastic  sequential  game  is 
introduced.  In  Section  3  the  definitions  of  admissible  strategy  pair,  perfect  equilibrium,  and  efficient 
strategy  pair  are  given.  Section  4  is  concerned  with  the  extension  of  Radner’s  approach  for  the 
construction  of  efficient  collusive  equilibria,  to  the  class  of  stochastic  sequential  games. 


2.  The  stochastic  sequential  game  format 


We  consider  a  two- person  noiuero-sum  discrete- time  dynamic  stochastic  game,  also  referred  to  as  a 
stochastic  sequentiol  game  and  defined  as  follows: 

Let  a  dynamic  system  be  described  by  the  state  equation 

*(<  +  !)  =  /(*(<).  »»(<).«,(t).w(t)),  f.0,1,’2,...  (1) 

where  z(f)  €  IR"  is  the  state  vector,  o<(l)  €  E.'"'  is  the  control  variable  of  Player  i,  i  =  1,2,  and 
{ui(t)  :  t  s  0, 1,2, . . .}  is  a  purely  random  sequence  of  independent  identically  distributed  random 
variables,  with  values  in  E';  n,  mi,  m},p  are  given  integers.  The  function  /  :  E''xE'"'  xE""’  xE'  — 
E”  is  given  and  known  by  the  two  players.  We  assume  that  each  player  is  able  to  directly  observe  the 
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atat«  vector  s(t),  but  can  observe  neither  his  opponent’s  actions  nor  the  realizations  of  the  random 
variable  )•  As  a  consequence,  Player  t  selects  his  control  ui(_t)  from  a  given  subset  Ui  of  1R’”‘ ,  m 
the  basis  of  the  information  represented  by  the  random  sequence 

v(t)  =  {*(0),i(l),...,i(t-l),z(t)},  t  =  0,1,2,...  (2) 

In  other  words,  a  strategy  of  Player  i  is  defined  as  a  sequence  of  mappings 

:  <*0,1,2,. »*1,2  (3) 

where  ■yn  associates  an  element  of  Ui  with  every  The  collection  of  all  strategies  of  Player  t  is 
called  Player  t’s  strategy  space  and  it  is  denoted  by  Fi. 

One  can  view  the  controlled  stochastic  system  (1)  as  a  &mily  of  discrete-time  stochastic  processes 
with  values  in  IR”,  defined  over  a  measure  space  ((!,£).  The  information  structure  (2)  corresponds 
to  an  increasing  family  of  o-fields  E  =  {E,  ;  t  =  0, 1, 2, . . .}.  A  strategy  yi  of  Player  t  is  a  E, -adapted 
stochastic  process  with  value  in  Ut-  Associated  with  any  admissible  strategy  pair  7  =  (yi,yj),  a 
probability  measure  Pj  is  defined  over  (0,  E). 

For  obvious  reasons,  a  strategy  requiring  a  player  to  recall  the  whole  sequence  i/(t)  for  every  t,  would 
be  of  little  practical  value.  Therefore,  we  consider  among  the  admissible  strategies  F]  the  class  of  the 
so-called  Extended  Markovian  (EM)  strategies.  Under  the  £3/ strategies  Player  i  chooses  his  control 
«,(<)  on  the  basis  of  an  extended  state  vector  S|(f)  =  where  p/(t)  is  an  auxiliary  state 

variable  with  values  in  a  given  set  Pi  and  which  summarizes  the  information  available  to  player  i 
concerning  the  history  of  the  game  up  to  time  t.  We  call  Z  *  IR."  x  x  y*  the  set  of  all  possible 
values  for  the  extended  state  variable.  The  evolution  over  time  of  the  auxiliary  state  variable  yt(t)  is 
described  by  an  auxiliary  state  equation  of  the  form 

»i(0)  =  y?. 

yi(<  +  l)  =  Si[*(0.“i(<).»a(<),«'(<).yi(<)],  <  =  0,1,2,...  (4) 

where  ji  :  IR"  X  ffi,'"'  x  in'"’  x  Dl'  x  Vi  —  y,  is  a  given  function,  and  y,?  is  a  given  initial  value. 

A  stationary  EM  strategy  7^  in  F,  is  such  that  for  every  <,  7,'«  is  a  function  of  the  extended  state  alone, 
i-e.,  7,1  does  not  explicitly  depend  on  t.  In  such  a  case,  the  symbol  y/  will  be  used  to  denote  y,-,,  i.e.. 
Player  I’s  decision  rule  at  stage  t,  as  well  as  his  strategy,  Le.,  the  whole  infinite  sequence  of  those  rules 
Also  one  should  notice  that  an  EM  strategy  implies  a  specific  information  etru^tare  associated  with 
the  auxiliary  state  equation  (4). 

Remark  1.  Tie  choice  of  tie  eiudliery  verieble  yi(t)  tad  state  equation  (4)  it  pert  of  tie  design  oi 
a  strategy  by  Player  1.  In  some  strategy  designs  tie  auxiUary  state  vs^riable  pi(t)  can  be  used  as  an 
indicator  of  tie  mood  of  play.  yi(f)  s  1  indicates  that  a  cooperative  mood  of  play  prevails  whereas 
IftCt)  =  0  indicates  that  a  noncooperatjve  mood  of  play  is  adopted,  in  seetjon  4  cooperative  equilibria 
will  be  obtained  in  this  class  of  EM  sUateipet. 

Remark  2.  The  to-coiled  stationary  feedback  strategies  for  which  the  extended  state  varitible  at  any 
time  t  reduces  to  s(i)  =  z(t)  constitute  a  particular  tubelatt  of  EM  strategies.  This  clast  of  strategfas 
has  been  tie  object  of  most  of  the  attentioa  devoted  to  the  theory  of  stochastic  sequential  games 
(Sobel  1971,  Whitt  1960). 
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3  Strategy  evaluation  criteria  (payoff  functionala)  and  equilibria 

Let  hi  :  Ui  X  Ut  X  TR."  IR.  represent  the  transition  reward  function  of  Player  i,  i  =  1,2.  We 
say  that  the  strategy  7  is  admissible  if  the  following  payoff  functionals  are  well  dehned  for  any  initial 
extended  state  r(0) 

(•T-i  1 

•^■[*(0):  7l  =  I  53  *‘(*(0.  (5) 

In  the  above  formula,  x,  ,  Ui ,  U}  are  stochastic  processes  associated  with  the  strategy  pair  7  and  the 
expectation  is  taken  with  respect  to  the  probability  measure  induced  by  the  strategy  pair  7,  where 

7  =  (71. 7j)  =  {71  =  (711, 7ji)  :  t  =  0,1,2,...}  (6) 

The  expression  (6)  defines  the  so-called  limit  of  average  criteria  for  strategy  evaluation  in  this  infinite 
horizon  stochastic  sequential  game. 

Remark  3.  The  use  of  a  limit  of  average  criterion  implies  that  the  player  is  only  concerned  by  the 
lasting  effects  of  his  strategic  choices.  More  precisely  any  effect  which  appears  in  a  finite  number  ot 
transitions  will  become  negligible. 

Remark  4.  With  the  limit  of  average  criterion  it  often  happens  that  the  payoffs  associated  with  a 
stationary  feedback  strategies  do  not  depend  on  the  initial  state.  This  ergodieity  property  will  be 
exploited  in  the  construction  of  cooperative  equilibria. 

Definition  1.  An  EM  strategy  pair  7"  =  (7},  73)  ,  associated  with  the  extended  state  z(t)  and  the 
auxiliary  state  equation  (4),  is  a  (perfect)  equilibrium  if  it  is  admissible  and  at  any  initial  extended 
state  r(0)  e  IR"  the  following  holds 


■/iW0);7-l>/;W0);7i.7;]  (7) 

for  all  fi  6  Pi  such  that  (71 , 73 )  is  admissible  and 

.73(r(0);7*)>-f,W0);7r,7»l  (8) 

for  all  73  6  Ts  such  that  (71,73)  is  admissible. 

Definition  2.  An  EM  strategy  pair  7*  s  (7]’, 73)  is  efficient  if  it  is  admissible  and  at  any  initial 
extended  state  s(0)  €  IR"  the  following  bolds  (or  any  7  idmissible 


^iW0);7l  >  .f<W0):7'l  <  =  1,2  =>  /,(r(0);7j  =  /tW0);7-]  «  =  1.2 


(9) 


4.  Efficient  Cooperative  Equilibria 


In  this  section  we  extend  to  a  sequential  game  format  the  approach  initially  proposed  by  Radner 
1981  in  the  realm  of  repeated  games,  for  the  construction  of  efficient  perfect  equilibria.  The  following 
assumptions  are  assumed  to  hold: 

(Al)  The  game  has  a  stationary  feedback  equilibrium  n  =  [fix, lit)  generating  payoffs  whose  values 
are  independent  of  the  initial  state  of  the  system,  i.e., 

=  const,  for  every  Zo  6  IR",  i=l,2  (10) 

In  addition,  there  exists  an  efficient  stationary  feedback  strategy  q  =  (qi ,  q: )  such  that 

=  VP  -  const,  for  every  Xo  £  nt",  i=  1,2  (11) 

and 

>  Vj"  fori  =  1,2.  (12) 

(A2)  There  exist  numbers  and  A/j  such  that 

I  M*. ’l.  W)  I  <  Af,  (1+ II  r  II),  i  =  l,2  (13) 

II  /[*.  »?(*).  w]  II  <  A/,(l+  II  w  II)  (14) 

for  every  z  £  AT. 

(A3)  The  components  of  the  random  vector  w  have  finite  expected  values  and  variances. 

The  random  variables  defined  below  will  serve  as  statistics  for  monitoring  adherence  to  cooperative 
policies  q  during  the  play.  Let 

1.7(0  =  /[i(«  -  1).  V{x{t  -  1)).  tu(t  -  1)1  (15) 

and 

«i(f)  =  f:{fii(z,(0. '?(z,(0)l*(<- 1))}-  »  =  1,2.  (16) 

Consider  the  stochastic  processes 

*i(0  =  Ai[z(t),  qi(z(t)),ui(f))]  -  ei(t) 
and 

»j(0  =  fii[z(0.  tii(0).'?j(*{0)]  -e»(0.  (!') 

and  define 

t 

5<(0*L  «<(*).  ‘  =  1.2 .  i=l,2  (18) 

$m0 

We  shall  denote  by  it(t)  and  l,(t)  the  values  of  ('■(0  and  Si(t)  respectively,  corresponding  to  the 
case  when  uy(t)  =  (*(<))  for  all  t  and  j  ^  i. 
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Remark  5.  Tlie  random  variable  x,(t)  defined  in  (15)  is  tbe  stats  tbat  would  result  from  the  use  of 
the  stntegy  pvt  r/  at  period  i  ~  1  and  at  state  s(t  —  1)  (teeell  thet  t;  is  a  feedback  strategy  peir). 
Tbe  coaditioael  expected  value  ei(t),  given  tbe  observed  state  r(t  -  1)  defined  in  (16)  can  tbus  be 
computed  by  Player  i  at  each  period  t.  Tbe  stochastic  process  9i(t)  defined  in  (17)  is  tbus  based 
on  a  comparison  between  tbe  conditional  expected  transition  reward  when  tbe  cooperative  feedback 
strategy  pair  prevails  and  tbe  actual  realization  of  this  reward.  Tbit  will  provide  tbe  information  basis 
permitting  Player  i  to  detect  cheating  by  bis  opponent  provided  tbat  be  can  observe  bis  own  transition 
rewards. 

Lemma  1.  Under  (A2)  and  (A3),  Si(t)/t  converges  to  zero  almost  surely. 

Proof:  This  result  is  a  direct  comequence  of  the  generalized  Strong  Law  of  Large  Numbers  (Feller 
1971,  page  243,  Theorem  3),  provided  that  £{$i(t)^}  can  be  shown  to  be  bounded  for  all  t  €  {1,2, . . .} 

To  see  that  the  latter  is  true,  observe  that 

£{$,(!)’}  =  i;{b,(*,(0.»b(x,(t))p}-c.(0’  .  (19) 

In  view  of  (A2)  one  has 

I  I  <  Afi(l+ l|x(<)ll! 

*  Wi[l+  II  /(*(<  -  1).  <7(x(t  -  !)).«'(«  -  1)1  111 

<  Aft[l  +  M,(l+  ||u.{t-l)|l) 

=  ilf[l+ 11  w(t- 1)111 

where  Af  is  a  constant  depending  on  Af,  and  Af}.  Hence, 

I  «<(«)  1=1  £{bi}  (  <  Af[l  +  £{||  w(t  -  1)  11)1  (21) 

and 

£{b?}  <  Af’[l  +  2£{||  u.(t  -  1)  ID  +  £{||  u:(t  -  1)  II’}]  (22) 

Therefore,  (A3)  implies  that  the  variances  of  $j(t)  are  bounded  for  all  t.  • 


We  now  proceed  to  the  construction  of  an  efficient  cooperative  equilibrium  defined  by  an  EM  strategy 
pair.  Let  {b((t)},  i  =  1,2  be  two  sequences  of  positive  numbers  such  that  bi(t)  tends  to  infinity, 
and  bi(<)/t  converges  to  zero  when  t  approaches  infinity.  We  define  tbe  auxiliary  state  variables, 
p<((),  t  =  1,2,  with  value  in  V)  =  {0,1}  and  tbe  following  dynamics 

y<(0)  =  1 

=  (  1  if  »<(<  -  1)  =  1  wd  Si(t)/t  >  -6,(f)/f  (23) 

1 0  otherwise 

we  define  an  EM  strategy  pair,  7  *  (7i ,  7j),  as  follows 


7((*(<).  »i(<).  »j(0)  = 


{ 


»?<(*(*)) 

i»i(*(0) 


if  y^(t)  =  I  for  j  =  1,2 
otherwise 


(24) 


Remark  0.  Tbe  dynamics  for  the  information  variable  yi(t)  given  by  £q.  (23)  does  not  seem  at  first 
sight  to  be  of  tbe  general  form  described  in  Eq.  (4).  However  it  would  be  easy  and  straightforward  to 
obtain  tbe  form  given  in  (4)  by  noticing  tbat  tbe  variables  S^t)  satisfy  tbe  following  state  equation 

Si(t+l)wS,(t)  +  *i(t)  ,t-l,2. 
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R«m«rk  7.  Xii«  itt»tegy  7<  ia  such  thtt  Player  i  bagins  the  pUy  In  •  cooperative  (Pareto  efficient) 
mood  ot  play;  however  it  aoae  "ciieeting”  ia  detected  at  period  t  through  the  mechaniam  defined  by 
(IT),  (18)  and  (23),  then  Player  i  switches  to  a  purely  noneooperative  mood  of  play  (feedback  Nash 
equilibrium)  and  maintaina  it  forever. 


Propoeition  1.  There  exists  a  sequence  {b(t)}  sucii  tJiat  the  strategy  pair  7  defined  by  (24)-(25) 
constitutes  e  perfect  equilibrium,  and  the  payoSs  generated  by  7  coincide  with  Vf ,  i.e.,  the  equiUbrium 
7  is  also  efficient. 

Proof:  If  yi(t)  or  V](t)  is  zero,  then  both  players  use  policies  (pi.p}),  which  constitute  an  equilibrium 
by  (Al).  Now,  consider  the  cases  when  pi  (t)  =  yj(t)  =  1.  Since  the  policies  (qi ,  72)  are  Pareto-optimal, 
any  deviation  from  q,  which  leads  to  an  increase  in  the  payoff  of  player  i,  must  at  the  same  time  cause 
a  decrease  in  the  payoff  of  the  other  player.  Suppose  that  Player  1  has  unilaterally  changed  for  a 
policy  Cl  which  generates  a  sequence  of  one  step  payoffs  which  satisfy: 


u,(t)],u,(t)}  =  £,{/», qi(i,(t)),qj(x,{t))]}  +  «i(t)  (25) 

If  the  above  change  of  policy  is  to  have  any  effect  on  the  overall  payoff  of  player  1,  one  must  have 


to(i/r)  i;  «,(t)  >  «,  >  0 


for  some  number  Si ,  because  otherwise 


^lim  (1/T)£«..„)  hi[x(l),  u,(t),«,(t)ll  *  ^bjn  (1/T)£  {£'  .■i.[x,(l),  q(x,(l))]} 

N  tsO  '  ^  taO  ' 

+  lim  (l/X)  £  Si(t) 

TweCO  ^ 

isO 

r-i 

<  V<^  +  Urn  (1/T)  £  Si  <  Vi<=  (27) 

.-D 

In  the  case  when  (26)  holds.  Player  2  will  receive  a  payoff  of  the  form 

^«..,.){'‘j(»(0.tti(«).u,(l)I}  »  r,{h,[x,(t),  tKx,(1))]}  +  «,(t)  (28) 


lim  (l/T)  £  <,{t)  <  «,  <  0  (29) 

1.0 

for  some  number  S2.  Hence,  in  view  of  Lemma  1, 

^lim  Sj(t)/t  <  UmSt(i)/t  +  St  <  62  aj.  (30) 

Since  &](t)/f  approaches  zero  when  t  tends  to  infinity,  (30)  implies  that  s/j(l)  will  eventually  become 
zero  almost  surely.  In  other  words,  any  deviation  irom  %  which  could  lead  to  an  increase  in  Player 
I’l  payoff  will  almost  surely  be  detected  by  Player  2,  who  will  then  twitch  to  p2-  The  best  response 
of  Player  1  in  such  case  will  be  to  switch  to  pi ,  and  hit  payoff  resulting  from  this  turn  of  events  will 


b«  V’,'''.  To  complete  the  proof,  it  suffices  to  show  that,  provided  th&t  the  sequences  {6j(t)) ,  t  =  1,2 
be  conveniently  deiined,  if  Player  1  stays  with  .  then  he  will  receive  ,  which  by  assumption  is 
greater  than  Vj". 

By  Lemma  1  we  know  that  for  every  integer  m,  there  exists  a  number  Tm  such  that 

for  f>r*}  =  l,  i  =  l,2  (31) 

Hence,  if  we  define  the  sequences  {h((t)}  t  =  1,2  in  such  a  way  that  they  satisfy 

4,(t)aoo  for  l<<<r,,  6<(t)  =  t/n»  for  Tm<t<T„^.l,  m  =  l,2,...  (32) 

then  Si(,t)/t  >  -bi{t)/t  almost  surely  for  all  (  and  t  =  1,2.  Therefore,  assuming  that  Player  2  uses 
jt,  if  Player  1  does  not  deviate  fiom  ih>  tken  yi(t)  and  yi(t)  will  almost  surely  remain  equal  to  one 
for  all  t,  which  means  that  Player  I’s  expected  payoff  corresponding  to  the  use  of  strategies  (24)-(25) 
will  be  equal  to  Vf . 

Since  the  same  argument  as  the  one  given  above  applies  to  the  analysis  of  consequences  of  deviations 
from  7]  by  Player  2,  we  have  shown  that  the  strategy  pair  given  by  the  expression  (25)  with  6,’s 
satisfying  (32)  is  in  fact  a  perfect  equilibrium  and  that  it  generates  Pareto-optimal  payoffs.  Thus,  the 
proof  has  been  completed.  • 


5.  Conclusion 

We  have  shown  that  a  class  of  sequential  games  with  limit  of  averages  payofis  admit  efficient  collusive 
equilibria  in  the  class  of  extended  markovian  strategies.  This  result  completes  and  extends  Radner’s 
theory. 
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1.  inKgUwctlgn 

Economic  Agents  operating  in  uncertain,  stochastic  environments  can  face  a 
tradeoff  between  current  period  expected  reward  and  accumulation  of  information  of 
uncertain  value.  For  example,  a  firm  producing  to  meet  uncertain  destand  might 
produce  at  the  expected  current  reward  isaximlzlng  output,  based  on  his  current 
beliefs  about  the  form  of  the  deaiand  curve,  or  It  might  choose  to  experiment  by 
varying  output,  thus  taking  short  term  losses  in  order  to  sharpen  beliefs  about  the 
form  of  the  demand  curve.  A  parametric  representation  of  the  agent’s  problem  is 
made  by  considering  the  utility  function  u(x,y)  and  the  conditional  density 
f(y|x,#}.  Here  the  random  variable  y  is  what  the  agent  is  trying  to  control 
(e.g.,  current  period  profits)  and  x  is  the  control  variable.  The  parameters  $ 
of  the  conditional  density  of  y  given  x  ate  unknown,  but  the  agent  has  opinions 
about  B  given  by  a  distribution  p.  The  agent  attempts  to  minimize  the  present 
discounted  value  of  the  stream  of  expected  losses,  EEf ^u(x^,y^) ,  where  the  expec¬ 
tation  Is  taken  with  respect  to  current  beliefs.  The  problem  Is  complicated  by  the 
fact  that  beliefs  are  updated  from  period  to  period  using  Bayes  Rule;  consequently 
current  period  actions  can  be  expected  to  influence  future  period  beliefs.  This 
introduces  stochastic  dynamics  Into  the  model. 

This  paper  considers  the  problem  In  the  case  in  which  the  density  f(y|x,F}  is 
a  location  family.  In  this  case  the  model  can  be  written  y  -  g(x,^)  c,  where  c 
is  an  1.1. d.  random  variable  whose  distribution  may  involve  unknown  parameters. 
When  g(x,N)  *  x'^  the  problem  is  one  of  controlling  a  linear  regression  process 
with  unknown  parameters  over  an  Infinite  horizon.  Many  approximate  control  rules 
for  this  problem  have  been  proposed,  for  example  sequential  least- squares  estimation 
combined  with  one -period  optimization  conditioning  on  the  current  estimates.  The 
analogous  policy  for  the  nonlinear  modal  Is  clear.  In  practice  several  policies  can 
work  "well,"  chough  It  Is  possible  to  compose  examples  In  which  the  policy  men- 
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tiomd,  fox  oxasplo,  if  fully  laprovfd.  ?roa  an  aeononle  awdalllns  point  of  viow, 
howfvfr,  wo  fXf  Intfroatad  in  tha  nnclul  policy,  and  in  the  eonsaquancas  for 
eonvarganca  of  baliafa  and  polleias  of  following  tha  opcinal  policy.  Vill  it  ba 
opclnal  for  an  agant  to  laam  tha  paxanatara  (and  cltus  eonvarga  to  "rational 
expaetationa")? 

Ibis  papar  giwaa  ganaral  eonditlona  undar  which  tha  saquanca  of  baliafa 
convaxgaa  to  a  linlt  and  tha  saquanea  of  optlawl  peliciaa  convaxgaa  to  a  linlt. 
Undar  furthar  condltiona  tha  linlt  policy  ia  tha  optimal  ona>pariod  policy  for  linlt 
baliafa.  Condltiona  undar  which  tha  limit  baliaf  ia  point  mua  at  true  parameter 
valuea,  correaponding  to  conalatant  parameter  eatinatea  are  more  atringent  and  are 
still  undar  inveatlgatlon. 

Laut-aquaraa  control  rules  In  the  linear  regression  model  have  been  widely 
discussed  and  studied  analytically  by  Taylor  (1974)  and  Jordan  (198S)  and  axperl* 
mentally  by  Anderson  and  Taylor  (1976).  Improvements  using  a  Bayesian  approach  ware 
suggested  by  Zellner  (1971)  and  studied  by  Harkana  (1975).  Tha  optimal  policy  in 
tha  linear  regression  case  hu  been  studied  by  iClefef  and  Nyarko  (1987) ,  who  obtain 
results  on  convergence  of  beliefs  and  policies,  convergence  in  a  different  elus  of 
models  hu  bean  studied  by  Easley  and  Kiefer  (1986).  Results  on  optimal  learning 
while  controlling  a  stochutlc  process  are  collected  along  with  an  example  in  Kiefer 
(1988). 

2.  The  Deelalon  Problem: _ Uncertainty.  Policies  and  Rewards 

In  this  section  we  sketch  the  general  framework  wo  wish  to  study. 

Lac  n'  be  a  complete  and  separable  metric  space,  let  7  be  its  Borel  field, 
and  (O',  ^  ,  F')  a  probability  space.  Define  the  stochutlc  process  on 

(O',  3'  ,  F').  The  are  usumed  to  be  independent  and  identically  distributed, 
with  the  common  urginal  distribution  p((g|f)  depending  on  some  parameter,  f  in 

R^,  which  is  unknown  to  the  agent.  We  usume  that  Che  set  of  probability  measures, 
(P('lf),  1*  continuous  in  the  parametar  (  (in  Che  weak  topology  of  meuuras);  and 

that  for  May  f,  J  c  p(d(|f)  >  0.  Lac  X,  the  action  snaea.  be  a  compact  subset 
of  R  .  Define  6  «  R  x  R  to  ba  the  parameter  space.  If  the  "true  paramecar* 
is  f  •  (d,f)  (  8,  and  the  agent  chooses  an  action  c  jt  at  data  c,  then  the 
agant  observes  y^,  where, 

♦  Cj.  (2.1) 

and  c  is  chosen  according  to  p('|0.  The  function  g  is  usumed  Muuxabla; 


further  reetrietions  are  latroduead  loplleltly  throu^  assuaptions  on  tha  updating 
aquation  (2.2)  and  tha  reward  function  (2.3). 

One  axaaq>la  ia  tha  aiaqtla  linear  tagraaaion  aodal  with  unknown  slope  and 
intareapt  and  with  tha  independent  drawa  froa  the  nonul  distribution  with 

2  •  rr 

Bean  sero  and  variance  a.  In  that  axaaple  O'  is  &  ,  is  the  collection  of 

Borel  sets,  on  R*.  and  P'  is  the  infinite  product  of  independent  univariate 

2 

notnal  distributions  with  aeans  rare  and  eeaaon  variance  o  .  The  paraaetar  f  is 

2  -  1 
tha  variance  of  c,  o  .  The  action  space  Z  is  a  closed  interval  in  R  .  The 

2 

paraaeter  /3  <  R  consists  of  tha  slope  and  intercept  of  tha  regression.  The 
2  1 

space  6  is  R  x  R^. 

Let  D  be  the  Borel  field  of  d,  and  let  P(6)  be  the  set  of  all  probability 
measures  on  (6,  O  ).  Endow  P(e)  with  its  weak  topology,  and  note  that  P(8)  is 
then  a  complete  and  separable  metric  space  (see  e.g.,  Parthasarathy  (1967,  Ch.  II, 
Theorems  6.2  and  6.5)).  let  t  P(d)  be  the  prior  probability  on  the  paraaeter 
space,  with  finite  first  moment. 

The  agent  is  assumed  to  use  Bayes  rules  to  update  the  prior  probability  at  each 
date  after  any  observation  of  (x^,  y^).  For  example,  in  the  initial  period,  data 
1,  the  prior  distribution  Is  updated  after  the  agent  chooses  an  action  x^,  and 
observes  the  value  of  y^.  The  updated  prior,  i.e.,  the  posterior,  is  then 

-  r(Xj,  yj^,  Mq).  where  r:X  x  R^  x  P(8)  -»  P(9)  represents  the  Bayes  rule 
operator.  If  the  prior,  has  a  density  function,  then  the  posterior  may  be 

easily  computed.  In  general,  the  Bayes  rule  operator  may  be  defined  by  appealing  to 
tha  existence  of  certain  conditional  probabilities,  although  some  care  is  needed 
(see  Diaconis  and  Freedman  (1986)).  Under  some  conditions  the  operator  T  is 
continuous  in  its  arguments,  and  we  assume  this  throughout.  Any  (x^,  y^)  process 
will  therefore  result  in  a  posterior  process,  where  for  all  t  -  1,2,..., 

Mg  -  r(x^,  Xg.  Mg.i)  (2.2) 

n-1  _  1 

Let  H  -  ?(6)  X  n  [X  X  R'*’  X  F(e)].  A  partial  history,  h  ,  at  date  n  is 
1-1  " 

say  element  h^^  -  (Mq,(Xj^,  yj^.  Mj^) . . . .  (x^^  j,  ^  \  '*• 

admissible  if  (2.2)  holds  for  all  t  -  1.2,...,  n>l.  Lot  H  be  tha  subset  of  H 

n  ti 

consisting  of  all  admissible  partial  histories  at  date  n.  A  policy  is  a  sequence 
s  •  where  for  each  t  k  1,  the  policy  function  -  jt  specifies  the 

date  t  action  -  Xg(h^),  as  a  Borel  function  of  the  partial  history,  h^  in 


/ 


162 


Hj.,  «t  that  data.  A  policy  fraction  la  atatlonarv  If  -  g(Mj)  for  aach  t, 

whara  cha  fraction  g(»)  aapa  P(6)  Into  i. 

Daflna  (Q.  5  ,  P)  -  (6,  .7  ,  x  (Q*.  5*  ,  P').  Any  policy.  »,  than 

ganarataa  a  saquanca  of  random  varlablao  {(x^(u),  y^(<>>).  ^ 

as  dascrlhad  abova,  using  (2.1)  and  (2.2).  Saa  Klafar  and  Ityarko  (1987)  for 

tochnlcal  datalla. 

For  any  n  -  1,2. .  let  3  ba  Cha  aub>flald  of  3  ,  ganaratad  by  the 

^  n 

random  varlablaa  (h  ,  x  ) .  Noclca  Chat  x  la  .?  •maasurabla  but  y  and  ^ 
n  n  n  n  n  n 

ara  not  3  -  maasurabla.  Naxt  daflna  5  -  V*.5_  . 

n  •  n**u  n 

Lac  u:X  x  -•  ba  tha  utility  function,  so  u(x^,  y^)  Is  tha  utility  Co 

tha  aganc  whan  action  Is  ehosan  at  data  C  and  cha  obsarvaClon  y^  Is  mads. 

The  reward  function  r:X  x  P(6)  R^,  Is  defined  by 

Tha  limer  Integration  marginalizes  with  caspact  to  c,  given  Che  parameter  (,  the 
outer  Integration  Is  with  respect  to  parameters.  Assume  that  the  reward  fraction  Is 
uniformly  bounded,  continuously,  and  concave  In  x  for  given  ft.  Kota  that  this 
assumption  restricts  g(*,*).  U(*,<)  and  p(*|«). 

Let  6  In  [0,1)  be  Cha  discount  factor.  Any  policy  x  generates  a  sum  of 
expected  discounted  rewards  equal  Co 


/  2  S^'\(xAu).  a  (u))P(d«) 
t-1  ' 


(2.4) 


where  the  (x^,  fi^  processes  are  Chose  obtained  using  the  policy  x.  A  policy  x 
Is  said  to  be  an  optimal  policy  If  for  all  policies  x  and  all  priors  ft^  in 

In  P(6),  V  ^(Mq)  >  V  (Mq).  Evan  Chough  cha  optimal  policy,  ir*  (when  It  exists) 

X 

nay  not  be  unique,  tha  value  function  V(^q)  •  V  ^(Mq)  1*  always  well-defined. 

X 


Straightforward  dynamic  programming  arguments  can  be  used  to  show  chat  station¬ 
ary  optimal  policies  exist  and  the  value  fraction  Is  continuous. 


Theorem  3.1:  A  stationary  optimal  policy  g:P(0)  X  exists.  The  value 
function,  V,  is  eoncinuous  on  P(9),  and  the  following  fractional  equation 
holds: 
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V(m)  -  max  (r(x.  <*)  +  «/V(?)p(d«U)M(6#))  (3.1) 

vh*r«  JI  -  r(x,  y,  m)  and  y  •  g(x,  fi)  +  t,  and  wfaara  tha  tncagral  la  takan  over 

X  6. 

Proof:  Let  S  -  {£:F(6)  •*  R  |  £  la  contlnuoua  and  bounded). 

Define  T:S  ->5  by 

Tw(^i)  -  max  (r(x,/I)  +  jJ‘v(M)p(d<|d)M(df))  (3.2) 

xcX 

One  can  eaally  show  chat  for  wcS,  TwcS;  and  chat  T  la  a  conccacclon  napping. 
Hence  there  exists  a  vcS  such  that  v»Tv.  Replacing  w  with  v  In  (3.2)  then 

results  In  (3.1);  and  since  vcS,  v  la  continuous.  Finally,  It  Is  Inoedlate  chat 

Che  solution  to  the  aaxlnlzaClon  exercise  In  (3.'2)  (replacing  w  with  v)  results 
In  a  stationary  opclaal  policy  function  (see  Blackwell  (1965)  or  Halcra  (1968)  for 
Che  details  of  Che  above  argunents) . 

4.  Convergence  of  the  Process  (m^)- 

In  this  section  we  prove  chat  the  posterior  process  converges  for  P>a.e  u  In 
(1,  to  a  well-defined  probability  naasure  (with  the  convergence  taking  place  In  a 
weak  topology) . 

Note  that  for  any  Borcl  subset,  D,  of  the  paraoeter  space  6,  If  we  suppress 

the  u's  and  let,  for  sons  fixed  w,  ;(^(D)  represent  the  mass  that  naasure  m^(w) 

assigns  to  the  set  0,  then 

Define  a  naasure  on  6  by  setting,  for  each  Borel  set  D  In  6, 

The  neasure  Is  Che  Uniting  posterior  distribution  and  Is  indeed  a  well-defined 
probability  neasure. 

Theoren  4.1.  Tha  posterior  process  converges ,. for  P-a.e.  u  In  Q, 

In  the  weak  topology,  to  the  probability  neasure 

Sunnarv  of  Proof:  Use  (4.1)  ebove  to  show  that  for  any  Borel  set  D  In  6, 

* 

M^(D)  Is  a  Martingale  neasure,  establish  that  Che  sequence  of  probability 
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Muures,  for  fixed  w,  is  using  the  Msusption  thst  the  first 

■oment  of  11^  is  finite,  then  spply  Prohorov's  Theorea  (e.g. ,  Billingsley 
(1968,  Theorem  6.1))  to  deduce  thst  is  e  probebility  meesure. 

Note  thst  this  result  on  convergence  of  beliefs  is  quite  different  from  the 
sbenderd  consistency  result  looked  for  in  econometrics.  The  Hsrcingele  Convergence 
Theorem  ellowe  us  to  esteblish  convergence,  but  the  limit  meesure  is  e  rendom 

verisble,  in  the  sense  thst  it  depends  on  Che  perticuler  sequence  of  shocks  reel- 
ized.  In  e  standard  estimation  problem,  the  limit  result  is  that  beliefs  converge 
and  Che  limit  belief  is  independent  of  sample  paths,  end  the  limit  belief  is  correct 
in  Che  sense  that  assigns  point  mass  to  the  true  parameter  value.  Standard 

results  do  not  hold  here  because  along  any  sample  path  for  which  beliefs  converge, 
Che  sequence  of  actions  (x^)  may  also  be  converging.  But  if  actions  converge  coo 

rapidly,  they  nay  not  generate  enough  information  to  identify  all  the  unknown 
parameters.  One  can  construct  examples  In  related  problems  in  which  this  phenomenon 
occurs  (see  e.g.,  Kiefer  (1988)). 

5.  Ontialratlon  and  Limit.  Beliefs  and  Actioi^s 

In  Theorem  4.1,  convergence  of  beliefs  was  established  for  an  arbitrary  (x^) 

sequence  (i.e.,  without  Caking  Into  account  the  underlying  maximization  problem). 

In  this  section  we  ask  what  action  (or  actions)  x  corresponds  to  the  limiting 
beliefs  «  . 

Theorem  5 . 1  establishes  chat  the  limit  action  Is  the  action  which  maximises 
single  period  reward  for  limit  beliefs. 

Theorem  5.1:  The  limit  action  x  >  Ilm  x  exists,  is  unique  for  given  ii) 

end  maximizes  the  one-period  reward,  r(x,M„).  for  limit  beliefs 

Proof  of  Theorem  5.1:  Recall  from  Theorem  4.1  that  lim  -  m.  exists  for 
ell  sample  paths.  The  sequence  (x^.)  end  satisfies  for  each  t 

(simultaneously,  a.e.)  the  functional  equation 

V(Mt)  -  r(x^.p^)  +  «JV(r(x^.y^,p^))p(dt|f)<i^(<W).  (5.1) 

Taking  limits  along  any  convergent  subsequence  gives 


-  r{x./*^)  +  *JV(r(x,y,^^))p(<U|{)M^(d#) 

vfa*r«  X  la  a  llait  point  of  tha  (x^)  aaquanca.  (In  taking  the  llalta  on#  uaea 

tha  fact  that  V  la  bounded  and  tha  Integral  In  (5.1)  la  E[V(p^)  to  apply 

Chung  (1974,  Thaozan  9.4.8).)  Howavec,  fcon  convergence  of  bellefa  (x,y)  ylelda 
no  Information  ao  r(x,y,/i^)  -  and  (5.1)  becoaaa  V(itJ)  -  r(x,M„)  + 

Now  wo  ahow  that  x  aolvea  the  problem 

max  r(x.M  )  (5.2) 

xcX 

Suppose  on  Che  contrary  chat  there  Is  an  xcX  such  that  r(x,/<  )  >  r{x,ii  ).  Then 
by  the  functional  equation 

V(;i,)  >  r(x.Mj  +  «J‘V(r(x.$.Mj)p<dt|e)p_(dO.  (5.3) 

But  by  Blackwell's  Theorem  (see  e.g.,  Klhlstrom  (1984,  Lemma  1,  p.  IS)),  since  Che 

experiment  "observe  (x,^)''  Is  trivially  sufficient  for  Che  experiment  ’make  no 
observations,*  we  obtain, 

;V(r(x,y,*iJ)p(d£|^)p^(dO  >  V(p  )  (5.4) 

Hence,  from  (5.3)  and  (5.4)  V(/i  )  >  r(x,M  )  SV{/t  ),  \^lch  is  a  contradiction. 

So  X  solves  problem  (5.2);  that  Is,  x  maximizes  the  one>perlod  reward  r(x,M) 
for  limit  beliefs,  n  .  Since  r(*,/i^)  Is  strictly  concave  In  x,  x  'must  be 
unique. 

6.  CanclMlon 

We  have  considered  the  decision  problem  facing  an  agent  controlling  a  nonlinear 

regression  process  when  parameters  In  the  mean  function  and  In  the  error  distribu¬ 
tion  are  unknown.  The  agent  faces  a  tradeoff  between  accumulating  Information  by 
varying  Che  values  of  the  regressors  and  accumulating  one-period  reward  by  following 
the  one-period  expected  reward  maximizing  policy.  Wo  show  that  Che  problem  can  be 
brought  into  Che  dynamic  programming  framework  and  chat  the  value  function  satisfies 
Che  usual  functional  equation.  The  sequence  of  beliefs  about  the  unknown  parameters 
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1«  (hown  CO  eonvmrg*  «l«o«e  ouroly.  Furthoc,  tho  opciaol  Action  pcoeoss  convocgos 
Co  the  one-period  opclnal  eeclon  under  llnle  beliefc. 
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INFORMATION  AND  DECISION  IN  OPTIMAL  INVENTORY  PROCESSES 
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ABSTRACT 

According  to  developments  In  management  Information  systems,  more  Investigation  Is 
required  to  adapt  the  fundamental  features  that  American  management  information  sys¬ 
tems  have  to  the  Japanese  technical  climate.  One  important  problem  is  to  decide  the 
kind  and  the  accuracy  of  management  information  systems.  If  complete  information  is 
desired  regarding  a  system  in  each  stage  of  control,  some  time  and  cost  will  be  en¬ 
tailed.  Otherwise,  if  incomplete  information  make  a  decision  quickly,  we  must  put 
up  with  using  a  probability  that  control  a  non-optimum  system.  We  have  not  the  com¬ 
plete  accuracy  for  the  information  and  the  decision  both.  This  is  analogous  to  Hei¬ 
senberg's  uncertainty  principle.  In  this  paper,  we  discuss  the  relation  between  the 
information  and  the  decision  in  optimal  Inventory  processes  in  this  viewpoint. 


INTRODUCTION 

According  to  management  information  system  development,  more  investigation  is  re¬ 
quired  before  adapting  the  funiamental  features  of  the  American  management  informa¬ 
tion  system  to  the  Japanese  technical  climate.  One  important  problem  is  to  decide 
the  types  and  the  accuracy  of  such  a  management  Information  system.  If  complete  in¬ 
formation  is  desired  regarding  a  system  in  each  stage  of  control,  some  time  and  cost 
will  be  entailed.  Otherwise,  if  incomplete  information  is  used  to  make  a  decision 
quickly,  we  must  put  up  with  using  a  probability  that  will  control  a  non-optimum 
system.  We  do  not  have  complete  accuracy  for  both  the  information  that  is  available 
and  decisions  that  are  made.  This  is  analogous  to  Heisenberg's  uncertainty  princi¬ 
ple.  This  paper  discusses  the  relation  between  information  and  decision  in  an  opti¬ 
mal  inventory  process  from  this  viewpoint. 

Additionally,  we  Introduce  the  general  principle  of  balance.  We  thus  possess  two 
weapons  namely  the  principle  of  optimality  in  dynamic  programming  and  the  principle 
of  balance  in  a  management  information  system.  In  the  third  section,  this  principle 
of  a  balance  will  be  applied  to  the  development  of  the  relation  between  information 
and  decision  in  optimal  inventory  processes.  Then,  problems  regarding  quantity  ap¬ 
proximation,  time  approximation,  demand  approximation,  the  criterion  approximation 
and  system  structure  approximation  are  summarized.  The  fourth  section  discusses  the 
stability  of  the  optimal  Inventory  equation  and  presents  a  design  for  an  optimal  in¬ 
ventory  system. 

Finally,  we  point  out  that  one  source  of  Imprecision  stems  from  both  randomness  and 
fuzziness,  and  conclude  with  a  discussion  of  some  areas  for  further  research. 
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Principle  of  Balance 


The  stochastic  properties  of  quantum  mechanics  are  based  on  the  uncertainty  princi¬ 
ple.  A  balance  relation  Is  pointed  out  wherein  It  Is  theoretically  Impossible  to 
measure  with  the  same  accuracy  at  the  same  time  for  a  pair  of  quantities,  called  a 
conjugate  quantity. 

^  The  phenomenon  that  Is  analogous  to  this  principle  In  physics  exists  In  many  fields. 

^  Let  us  generally  call  this  the  principle  of  balance  and  discuss  the  relationship  be¬ 

tween  this  principle  and  several  phenomena. 

For  example,  the  approximating  of  linear  prediction  theory  due  to  Wiener  leads  to 
the  problem  of  minimizing  the  quadratic  form 


N  M 

k«0 


over  the  real  quantities  ,  where  the  quantities  are  given  real  numbers.  E 

Is  the  prediction  error.  The  prediction  error  decrease  and  Che  structure  complex 
Increase  when  M  Is  Increased.  It  Is  an  important  practical  question  of  decide  how 
large  Co  make  M  chat  balance  the  prediction  error  and  the  structure  complex. 


(1)  Principle  of  Optimality  in  Dynamic  Progranming  [1] 

The  principle  of  optimality  In  dynamic  programming  Indicates  that  the  optimal  policy 
should  harmonize  Che  balance  between  costs  Involved  In  deciding  present  and  future 
values  on  a  new  state  reduced  by  Its  decision,  because  dynamic  programming  Involves 
multi-stage  decision  processes.  The  Information  for  the  future  Is  necessary  In  to 
make  a  decision  In  Che  present.  The  principle  of  optimality  is  an  exact  mathemati¬ 
cal  expression  for  this  Idea. 

Let  us  assume  Rth  multi-stage  decls'on  processes.  We  shall  be  concerned  with  cri¬ 
teria  possessing  a  structure  which  ,  irmits  us  to  focus  our  attention  solely  upon  the 
past  and  present  history  of  the  process  In  a  search  for  values  of  policies.  Then, 

Co  construct  the  optimal  policy  of  the  Rth  stage,  whatever  Che  Initial  state  and 
Initial  decision  are,  Che  remaining  (R  -  l)th  decisions  must  constitute  an  optimal 
policy  with  regard  to  the  state  resulting  from  the  decision  on  the  first  stage.  We 
must  determine  the  first  decision  In  order  Co  determine  the  balance  between  gain  In 
Che  first  stage  and  gains  In  remaining  (R  -  1}  stages. 


(2)  Principle  of  Balance  In  Information  and  Decision 

If  complete  knowledge  of  the  system  is  deemed  necessary  at  any  stage,  then  an  appre¬ 
ciable  time  is  usually  required  to  accumulate  this  data.  During  this  time,  Che  sys¬ 
tem  Is  onconcrolled .  That  Is  to  say  chat  time  Is  one  of  the  s»sc  valuable  resources 
we  have;  It  Is  unique  In  the  fact  that  It  cannot  be  reversed  or  replaced.  It  Cakes 
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Clme  to  make  decisions  and  Chen  to  Implement  those  decisions.  If,  however,  we  make 
a  decision  quickly,  using  Incomplete  Information  about  the  system,  there  is  a  non- 
negllglble  probability  Chat  a  non-optimal  action  will  be  taken.  We  cannot  have  com¬ 
plete  accuracy  In  both  Information  and  control.  This  is  the  uncertainty  principle 
In  a  management  Information  system. 


Information  and  Decision  in  Optimal  Inventory  Processes 

This  section  discusses  some  applications  of  principle  of  balance  in  regard  to  infor¬ 
mation  and  decision  in  multi-stage  stochastic  inventory  control  processes.  Multi¬ 
stage  stochastic  Inventory  control  processes  will  be  Introduced  In  Section  it.  At 
first.  If  we  observe  the  exact  Inventory  quantities,  then  we  have  the  right  decision 
and  the  optimum  expected  cost,  but  we  must  accordingly  allow  for  the  cost  of  more 
observation.  This  sort  of  approximation  relates  to  the  quantity  aspects.  Secondly, 
instead  of  keeping  records  and  placing  orders  at  each  period,  It  may  be  better  to 
observe  and  order  at  intervals  of  a  few  periods,  even  when  this  delay  necessitates 
paying  a  penalty  charge  for  getting  items  quickly.  This  type  of  approximation  re¬ 
lates  to  the  component  of  time.  Also,  there  are  some  approximation  problems  In  re¬ 
gard  to  determining  demand  information,  optimum  criterion  and  inventory  system  struc¬ 
ture,  etc. 


(1)  Approximation  of  Observation  [71,  [8] 

A  major  problem  In  modern  management  is  that  of  keeping  records.  However,  sometimes, 
at  a  certain  point,  the  cost  of  keeping  records  is  greater  than  the  gain  that  is  ob¬ 
tained  by  using  these  records.  These  factors  provide  the  motivation  for  a  study  of- 
the  approximation  of  observation  of  Inventory  quantity.  It  Is  necessary  to  decide 
on  the  degree  of  observation  approximation  that  harmonizes  with  the  observation  cost 
and  the  gain  obtained  by  using  approximate  information.  We  have  obtained  the  follow¬ 
ing  results,  using  both  analytic  and  computational  studies.  [7,  8] 

1)  Optimal  choices  between  degree  of  observation  M  and  degree  of  policy  N  depend 
on  the  unit  costs  for  this  inventory  process. 

2)  Inventory  processes  are  as  sensitive  to  M  as  to  N  . 

3)  Inventory  processes  are  as  sensitive  to  the  backlogging  problem  as  to  the  lost 
sales  probler,  etc. 

We  can  determine  the  degree  of  approximation  that  balances  the  cost  of  observation 
and  the  total  expected  cost.  If  the  approximate  observation  quantity  is  used. 

(2)  Approximation  in  Time  [9],  [10] 

In  introducing  the  basic  optimal  Inventory  equation,  explicit  use  was  made  of  the 


172 


assunptlon  that  observatlona  and  orders  are  made  at  each  period.  However,  this  aa- 
saoption  may  be  questionable.  Instead  of  keeping  records  In  every  period.  It  may  be 
better  to  count  the  number  of  Items  when  the  supply  Is  low,  and  even  to  pay  a  penalty 
charge  for  getting  Items  quickly  when  the  supply  Is  very  low.  The  problem  that  we 
want  to  study  is  that  of  determining  the  time  to  examine  the  number  of  Items  remain¬ 
ing  in  stock. 

The  results  of  analytic  and  computational  studies  are  given  In  [10] .  As  we  might 
have  expected,  the  shortage  and  the  total  expected  cost  Increase  with  Increasing  var¬ 
iability  of  time  Interval  In  decision.  Thus,  we  can  determine  the  time  of  observa¬ 
tion  and  control  that  balances  the  cost  of  observation  and  the  expected  cost,  which 
are  obtained  by  using  an  approximate  time. 


(3)  Approximation  of  Demand  Information  [11] 

The  first  step  away  from  completely  deterministic  demands  and  a  step  of  considerable 
import,  is  the  classical  theory  of  probability  with  Its  introduction  of  random  vari¬ 
ables.  We  want  to  Indicate  the  existence  of  high  levels  of  uncertainty.  We  can  con¬ 
sider  the  following  three  cases. 

1)  Stochastic  problem;  the  case  when  the  stochastic  feature  Is  known. 

2)  Adaptive  problem;  the  case  where  the  demand  distribution  contains  unknown  sto¬ 
chastic  parameters. 

3)  Game  theoretic  problem;  the  case  when  the  stochastic  feature  is  unknown. 

In  [11],  we  have  compared  the  solutions  for  cases  when  the  probability  density  func¬ 
tions  of  demand  are  assumed  to  be  exactly  known,  adaptively  known  and  game  theoreti¬ 
cally  known.  The  total  expected  cost  Increases  as  the  completeness  of  Information 
decreases. 


(4)  Approximation  in  Criterion  [12] 

The  problem  of  establishing  Che  Inventory  system  effectiveness  criterion  Is  a  very 
fundamental  one. 

Let  us  discuss  the  following  problems  of  a  multi-stage  nature,  namely  average  cost 
per  period  and  probability  criterion. 

1}  Multi-stage  problem 

We  often  say  that  we  are  planning  for  the  next  year,  and  that  we  wish  to  minimize  the 
multi-stage  expected  costs  or  maximize  the  multi-stage  expected  profits.  It  Is  clear 
Chat  managers  need  not  plan  for  next  year  only  and  that.  In  fact,  they  must  consider 
many  years  In  advance.  In  this  case,  the  optimal  policy  In  one  period  does  not  al¬ 
ways  mean  the  optimal  policy  In  multi-stage  periods.  However,  under  some  assumption, 
the  former  coincides  with  the  latter. 
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2)  Average  cost  per  period 

In  the  stationary  approach,  we  select  a  particular  (s,  S)  policy,  calculate  the 
long-run  costs  based  on  this  policy,  and  then  select  the  policy  variables  so  as  to 
mlnlailze  long-run  cost.  Let  the  mlnlnnun  cost  be  denoted  by  k  .  In  the  dynamic  pro¬ 
gramming  approach,  the  technique  depends  on  the  minimum  cost  function  .  If 

the  interest  rate  Is  zero,  then,  as  period  n  becomes  infinite,  C  (x)  will  tend 

a 

toward  Infinity.  It  seems  plausible  that  there  will  be  some  connection  between 

lim  (C^(x)]/n  and  k  . 
n-» 

3)  Probability  criterion  [i2l 

Let  us  discuss  the  criterion  which  minimizes  Che  probability  that  Che  inventory  over 
all  stages  exceeds  a  fixed  level.  The  profits  In  the  probability  criterion  are  as 
follow.  At  first,  this  is  simple,  because  we  do  not  require  an  estimation  of  the 
cost  functions.  Secondly,  we  have  the  same  policy  characterized  by  the  principle  of 
constant  stock  level  as  the  criterion  of  cost  functions. 


(5)  System  Structure  Approximation  [12] 

Consider  an  inventory  system  that  has  many  benefits.  Under  Individual  Inventory  con¬ 
trol,  each  location  puts  In  their  orders  separately  and  Is  concerned  only  with  Its 
own  welfare.  Under  its  centralized  inventory  control  procedure,  by  contrast,  quan¬ 
tity  orders  are  made  simultaneously  for  all  locations  in  the  network.  There  are  Im¬ 
mediate  advantages  and  disadvantages  to  controlling  such  an  inventory  system  central¬ 
ly. 

Since  Information  about  Che  entire  supply  network  Is  recorded  at  a  central  location, 
decisions  can  be  made  effectively  and  expediently  in  emergencies,  but  the  resulting 
decisions  are  more  complex.  An  Important  question  is  Che  determination  of  how  many 
benefits  that  are  optimal  In  order  to  achieve  centralized  control. 


Inventory  System  Design 

Most  of  the  authors  who  have  written  on  the  subject  of  inventory  control  have  made 
Che  assumptions  either  that  we  have  obtained  or  that  we  shall  have  Information  used 
to  make  the  necessary  decision.  In  this  section,  on  the  contrary,  let  us  determine 
the  kind  and  accuracy  of  the  Information,  on  the  assumption  chat  we  know  how  to  de¬ 
cide  when  Co  have  some  Information.  [3j 

There  are  two  types  of  costs  with  regard  Co  Information.  One  Is  the  observation 
cost,  which  is  entailed  In  obtaining  information.  The  other  Is  the  error  cost,  ow¬ 
ing  to  the  approximation  of  Information.  Our  aim  Is  to  minimize  the  sum  of  these 
costs.  A  model  of  our  Inventory  control  process  Is  the  multi-stage  stochastic  in¬ 
ventory  problem.  Let  L(y)  be  given  by: 
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•  L(y)  -  f^h(y-i)4a)dZ  +  J^p(e-y)(K5)de  (1) 

Where  L(y)  represents  the  sum  of  the  expected  Inventory  cost  jQh(y-C)4>(€)d€  and 

the  expected  penalty  cost  J^p(S-y)iKC)<li  in  each  period,  and  ip(()  Is  the  demand 

probability  density  In  each  period,  given  that,  at  the  beginning  of  a  period,  the  sum 

of  the  Initial  Inventory  on  hand  and  the  stock  to  be  received  In  a  period  Is  y  .  We 

define  the  functional  f  (x)  as  Che  total  expected  discounted  cost  over  n  periods, 
n 

where  x  Is  the  Inventory  on  hand  at  Che  beginning  of  the  first  period.  We  have  ob¬ 
served  that  can  be  written  for  all  x  In  the  following  functional  equation. 

f^(x)  »  min  [c(y-x)+L(y)+a^f^_j^(x-e)()i(S)d5]  (2) 

yix 

In  (2)  c(y)  Is  the  ordering  cost  and  a  denotes  Che  discount  factor  (D  <  a  <  1) . 
AC  first,  we  shall  review  the  stability  of  the  Inventory  process  which  Is  fundamental 
Co  design  Issues.  In  the  following,  we  Introduce  the  Inventory  processes  design  and 
the  Inventory  policies  control  problems. 

DISCUSSION 

We  have  shown  chat  the  environment  for  approximation  of  observations  and  policies 
affects  our  Inventory  processes. 

Much  of  Che  decision  making  In  the  real  world  cakes  place  In  an  environment  In  which 
Che  goals,  the  constraints  and  the  consequences  of  possible  actions  are  not  known 
precisely.  To  deal  quantitatively  with  imprecision,  the  traditional  approach  is  to 
employ  the  concepts  and  techniques  of  probability  theory  and,  more  particularly,  the 
tools  provided  by  decision ' theory,  control  theory  and  Information  theory.  This  Is 
now  questionable  especially  In  view  of  Che  developments  In  the  field  of  fuzzy  sets 
theory  [4],  [5]. 

There  Is  a  differentiation  between  randomness  and  fuzziness,  with  the  latter  being 
a  major  source  of  Imprecision  In  many  decision  processes.  By  fuzziness,  we  mean  a 
type  of  Impression  which  Is  associated  with  fuzzy  sets,  that  Is,  classes  in  which 
there  Is  no  sharp  transition  from  membership  to  nonmembership.  There  are  many  facets 
of  Che  theory  of  decision  making  In  a  fuzzy  environment  which  require  more  thorough 
Investigation.  This  Is  also  the  case  with  Inventory  systems. 
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ABSTRACT; 


This  paper  propound  a  preference  model  and  a  decision  model  for  an  economic  system.  With  the 
multicriterion  optimisation,  we  develop  a  method  permitting  the  choice  of  stock  portfolio  in  stock 
exchange.  In  the  first  part,  we  compare  two  fiizzy  numbers,  representative  of  the  value  of  two 
actions  according  to  a  criterion,  and  we  introduce  the  threshold  notion.  In  the  second  part,  we 
compare  two  classing  result  from  two  criterions ;  a  new  classing  will  be  deduct  of  this  comparison. 
This  one  give  the  composition  of  a  portfolio. 


I  INTRODUCTION 

The  stock  exchange  is  too  complex  economic  system  to  build  an  elaborate  model.  Nevertheless, 
with  a  simple  model,  we  can  have  a  better  understanding  of  the  system  ;  and  so  deduct  a  best 
adapted  command. 

There  is  two  types  of  models : 

-  Economic  model :  with  several  parameters  as  social,  political,  financial  events. 

-  Technical  model :  based  on  the  quotations,  these  models  give  the  value  of  the  Beta  coefficient,  the 
index .... 

The  first  one,  too  much  subjective,  must  be  used  with  a  lot  of  precautions.  The  second  one  contains 
a  part  of  the  information  wich  circulates  in  the  market.  The  estimations  given  by  the  technical 
models  are  often  biaised  ;  since  they  are  obtained  by  the  study  of  the  past.  It  is  important  to  utilize 
a  method  which  integrate  this  error. 

These  different  models  give  a  set  of  criterions  which  allow  to  valuate  the  stock  exchange. 

Let  C={C,,Cy. the  criterion  set. 

A=  {AjA2v.Aj}  the  possible  set  of  stocks  on  the  system 

To  integrate  the  error  of  estimation,  we  consider  an  uncertainty  domain  containing  the  value.  So  all 
the  values  become  fuzzy  numbers 
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II  COMPARISON  OF  TWO  FUZZY  NUMBER 

In  this  part,  we  develop  a  preference  model  allowing  the  comparison  of  two  fuzzy  numbers.  In  our 
^>plication,  each  criterion  is  applied  on  each  stock.  So  a  criterion  matrix  C,(aj)  can  be  created. 
With  the  notion  of  fuz^  numbers,  the  criterions  become : 

[Ci‘(aj),C/(aj)] ;  where  [C.C*]  represent  the  limits  of  a  uncertainty  intervaL  The  domain  of 
fuzzy  number,  representative  of  the  criterion,  integrate  the  error  of  estimation.  We  must  add  to  this 
error,  the  possibility  of  a  future  event  unforeseeable  by  the  past  For  that  reason,  we  introduce  a 
threshold  notion :  S,.  A  stock  (aj)  will.be  better  than  an  other  stock  (a^),  for  a  criterion  (C,),  if  the 
value  of  the  criterion  applied  to  the  stock  (a^)  is  better  than  the  value  obtained  with  the  second 
stock  (a^) ;  and  if  the  difference  is  higher  than  the  threshold  S.. 

If  Cj(aj)  >  Cj(a^)  +  Sj  =  >  is  prefered  to  a^  for  C, 

In  this  case,  a  perturbadon  P,  due  to  an  unforeseeable  event,  will  not  change  the  choice,  if  bis 
amplimde  is  not  higher  than  the  threshold  Sj. 
if  P  <  S.  we  have : 

Cjfa^)  >  C,(a^)  +  P  =  >  a^  is  always  prefered  to  a^ 

for  the  criterion  C,. 

We  see  immediately  than  if : 
q(aj)  <  C.(a,)  +  S, 
q(a,)  <  C.(a.)  +  S. 

Then,  3l  is  not  prefered  to  a^  and  a^  is  not  prefered  to  a..  To  describe  this  pwssibility,  we  introduce 
the  indifference’s  reladon ;  which  is  noted :  I . 

In  order,  to  compare  two  stocks  aj  and  a^  by  using  the  criterion  Cj,  we  have  consequently  three 
choice  reladons ; 

-  Preference  (P) 

•  Indifference  (I) 

-  No  preference  (NP) 

We  define  the  different  reladons  as  following : 

I  Ci+(aj)-Ci+(ak)  >  Sj 

II  Ci-(aj)-Cj-(a|j)  >  Sj  ->  ajPa^ 

12  Ci-(aj)-Ci-(ak)  <  =  Si 

Ginsideiing ; 

M  =  ((Ci+ (aj)  +  Ci-(aj)HCi+ (a^)  +  Ci-(ay,)))/2 
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We  can  use  a  function  of  M,  noted :  F.  This  function  (f)  depends  on  several  parameters  (uncertainty 
domain,  threshold, ...). 
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Representation  of  the  different  cases 


2  q+(a,)-C,+(a,)  < 

The  obtained  results  are  the  same  than  the  precedent,  if  we  change  j  and  k. 


3 

-Sj  <=  q+(aj)-q+(a^)  <  = 

Si 

31 

..S.  <  =  F  <  = 

s,  *> 

F  > 

Si  => 

F  < 

-S.  “  > 

a,NP 

The  method  consist  in  comparing  the  n  actions  two  by  two,  for  the  criterion  q.  A  preference  matrix 
Pc,(jjc)  (n*n  dimension)  can  be  buUt  To  use  this  matrix,  we  must  code  the  preference  relations. 

We  take  the  following  coding : 

if  P  a^  »>  Pc,(j.k)  =  1 

if  aj  I  a^  =>  PCi(j,k)  =  0 

if  a^  NP  *>  Pc,(j,k)  *-l 

Every  criterion  will  have  a  matrix  n*n  which  display  the  comparison  between  the  stocks. 

For  one  criterion,  two  stocks  are  compared  owing  to  the  rows  of  the  matrix  Pcj.  The  row  j  shows  the 
preference  for  the  stock  j  with  regard  to  the  others.  The  value  of  the  preference  for  this  stock  is 
given  by  the  sum  of  the  row. 

Let  us  build  the  vector : 

Vromi(k) »  PCi(i,k) 

Vsomj(k)  represent  a  performance  measure  of  the  stock  a^.  We  would  terminate  this  study,  if  the 
found  classificadon  was  the  same  for  all  criterion  q ;  this  is  improb^le.  So  we  introduce  a  method 
which  compare  two  contradictory  classing.  This  method  will  give  the  value  of  all  the  stocks  for  the 
criterions  set  and  then  a  classing. 


in  COMPARISON  BETWEEN  TWO  CnmERIONS 


In  this  part,  we  develop  a  decision  model  allowing  the  comparison  of  two  criterions.  Tliis  method 
consist  in  setting  the  criterions  in  a  hierarchical  order,  beginning  by  the  most  important  for  the 
investor,  and  endii^  by  the  less  one.  With  two  classing,  related  to  two  criterions,  we  build  a  new 
classing.  We  take  the  two  most  important  criterions,  then  the  second  and  third,  and  so  on.  To  be 
able  to  define  a  global  classing  with  two  contradictory  primary  classing,  we  introduce  the  criterion 
preference  coefficient,  written  CLj. 

Let  aj  and  a^  be  two  stocks,  Cj  and  be  two  criterions  with  prefered  to  q.i-  The  stocks,  aj 
and  have  each  one  a  value  for  each  criterion :  Vsomj(aj),Vsomj^j(aj)  for  the  stock  aj ;  and 
Vsomj(ajj),Vsomj^j(a|j)  for  the  stock  a^.  More  exactly,  we  normelize  the  vectors  Ysom^faj)  by  doing 
the  following  tranformation : 

Vsom.(a^)  =  (Vsom.(aj)-Vsomj  min)/(Vsomj  max-Vsomj  min) 

With  Vsomj  min  and  Vsomj  max,  the  minimum  and  the  maximum  value  of  the  vector  Vsom.(a.),  for 
the  criterion  C,. 

In  order,  to  establish  the  new  classing,  we  make  the  difference  between  the  two  values  of  the  two 
stocks  for  a  criterion : 

D.  =  Vsom.(aj)-Vsom;(a^) 
if  D.>0  =■>  ajPak 

if  0  =>  akPaj 

This  two  criterions  do  not  permit  us  to  choose  between  the  stocks  a^  and  a^.  A  solution  consist  in 
saying  that  the  second  criterion  is  less  important  than  the  first  one.  Then  we  multiply  the  second 
(Dj*,)  by  the  criterion  preference  coefScient  (CPj) ;  with  CPj  less  than  CP|^.j.  The  new  preference 
relation  becomes : 

So  we  have  the  following  relations : 

1)  Obvious  relations : 


Dj  >  OandDj^j 

o 

II 

A 

=  > 

ajP 

Dj  <  OandDj^, 

O 

II 

V 

*  > 

a,NP 

Dj  =  Oand  Dj^j 

*0 

=  > 

Dj  =  0  and  Dj,., 

>0 

=  > 

Dj  ■  OandDj,, 

<0 

■  > 

a-NP 
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2)  Solutions  depending  of  CPj : 

D,  >  0  and  D,^j  <  0  or  Dj  <  0  and  Dj^,  >  0 

Di  +  >  0  =>  a^Pa^ 

D.  +  D.^,*CPj  <  0  ->  ajNPak 

Di  +  Dui’CPi  *  0  ”>  ‘‘jK 

By  conq)aring  n  stocks  with  themselves,  we  obtain  a  preference  matrix  Pj  (n*n).  And  then,  we 
calculate  the  sum  vector  VSj(k) ;  it  gives  so  a  new  classing  which  use  the  two  criterions  C,  and  C,,.  j. 
With  the  criterion  C,,.,  and  C,,.y  we  obtain  the  sum  vector  VSj^j(k).  The  result  of  VS,  and 
will  be  compared  with  and  will  give  VSj^20^)  •  ^  least,the  sum  vector 

will  give  us  the  definitive  classing  for  the  criterions  set 

Exemple : 

let  A={A1,A2,A3} 


C={C1,C2} 

S-0 


Pci  = 


PCj. 

let  CL;=0 
P,  - 


=  >  Vsom,  = 


Vsomj 


■  >  VSi  - 
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IV  CONCLUSION 


The  use  of  this  method,  on  the  stocks  of  the  stock  exchaage  of  Paris,  has  a  real  increase  of  the  portfolio 
D'jfit  with  regard  to  the  method  of  simple  investing.  Our  work,  at  the  present  time,  is  oriented  to  the 
amelioration  of  these  models  ;  with  the  function  P,  as  also  on  the  determination  of  the  different 
thresholds. 
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STOCHASTIC  AND  QUANTUM  SYSTEMS 


Logarithmic  Transformations  with  Applications  in  Probability  and  Stochastic  Control 


Wendell  H.  Fleming 

Division  of  Applied  Mathematics,  Brown  University,  Providence, 
Rhode  Island  02912. 


Atstract 

We  arc  concerned  with  a  class  of  problems  described  in  a  somewhat  imprecise 
way  as  follows.  Consider  a  linear  operator  of  the  form  L  +  V(x),  where  L  is  the 
generator  of  some  Markov  process  Xj  and  the  "potential"  V(x)  is  some  real-valued 
function  on  the  state  space  of  X(.  We  are  interested  in  probabilistic  representations 
for  solutions  u(t,x)  of  the  evolution  equation 

(1)  ^  =  Lu  +  V(x)u,  t  >  0 

with  initial  data  at  t  =  0.  The  Feynman-Kac  formula  gives  a  well-known  stochastic 
representation  for  u(t.x).  We  seek  a  different  probabilistic  representation  for 
I  =-log  u,  if  u(t.x)  is  a  positive  soluUon  to  (1).  In  this  representation  the  operator 
L  is  replaced  by  another  generator  L,  (perhaps  time  dependent),  chosen  to  solve  a 
certain  stochastic  control  problem.  The  dynamic  programming  equation  for  this 
stochastic  contrql  problem  is 

(2)  gf  =  H(I)  -  V(x),  where 
H(l)  =  -c>L(c-'). 

Another  way  to  view  the  change  of  generator  from  L  to  is  by  change  of 
probability  measure  through  conditioning. 

Next  suppose  that  the  state  space  of  Xj  is  euclidean  R",  that 

L  =  L^,  u  =  u*,  I*  =  -  «  log  u*  and 
H^I)  = -c>L,(e-I). 

Under  various  assumptions  it  turns  out  that  I‘  -  I®  as  e  -  0, 
lim  =  Ho(x,Ix) 

where  Ix  is  the  gradient,  and  that  I(t,x)  is  a  viscosity  solution  of  the  first-order 
partial  differential  equation 

at  =  Ho(x,Ix). 


When  x,  is  a  nondegenerate  difhision  on  R",  then  L  is  a  second  order  elliptic 
partial  differential  operator.  In  this  case,  the  logarithmic  transmation  provides  an 
analytical  approach  to  large  deviations  questions  of  Ventsel'F  reidlin  type,  and  for 
more  precise  results  in  the  form  of  asymptotic  series  expansions  of  in  powers  of 
(.  The  logarithmic  transformation  technique  is  also  of  use  to  study  certain 
asymptotic  problems  in  which  u‘(tpc)  obeys  a  nonlinear  parabolic  partial  differential 
equation. 
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MACROSCOPIC  PROPERTIES 

Of 

DISCRETE  DIFFUSIONS 

by 

E.  Benoit* ,  B.  Candelpergher** ,  C  Lobry** 


The  mathematical  Wiener  process,  and  more  generally  mathematical 
diffusion  processes,  are  mathematical  idealizations  of  physical  processes  like  Brownian 
motion  for  instance  .  The  starting  point  in  the  definition  of  the  Wiener  process  is  the 
definition  of  the  random  walk  :  ^ 

?t+dt  ■  5t 

R.W.(dt)  t  6  {  0,  dt,  2dt,...j)dt . ndt  =  i } 

?o  =  0 

where  is  a  sequence  of  independent  random  variables  taking  values  ±i  with  equal 

probabilities  .  This  is  certainly  a  single  mathematical  object  but  not  a  good  idealization 
because  it  contains  a  parameter  dt  and  ,  i priori ,  there  is  no  universal  way  to  fix  it . 

As  it  is  well  known,  the  usual  way  to  deal  with  this  difficulty  is  to  consider 
the  whole  femilly  R.W.(dt)  (dt  >  0)  and,  because  we  want  to  Idealize  physical  random  motions 
in  which  the  elementary  step  is  very  small  compare  to  the  scale  where  the  phenomenum  is 
observed,  we  take  the  limit  when  dt  «  0  .  This  limit  is  a  mathematical  object  known  as  the 
Wiener  pmceee.  ilathematlcally  speaking  the  Wiener  process  is  a  probebillty  measure  on 
the  infinite  dimenslonel  vector  space  :  ^  ^ .  Because  of  the  great  cardinality  of  this  space 

a  probility  measure  is  no  longer  a  simple  abject  and  its  definition  requires  the  knowledge  of 
most  of  the  technicalities  of  measure  theory  .  The  only  wag  to  avoid  all  those  technicalities  Is 
to  work  only  on  probability  spaces  of  finite  cardinality  .  This  is  the  case  of  the  random  walk 
R.W.(dt)  which  is  defined  on  the  soece  f-1  .  -t-l  I  .  But  we  lose  universtiitu  of  the  wiener 


process  .  How  can  we  recover  it  ? 


Define  the  Wiener  Weik  (W.W)  as  R.W.(dt)  for  dt  infinftesimei  and 
look  for  its  mecroscopic  properties  . 

The  real  number  dt  considered  here  is  infinitesimal  in  the  fOneel  sense 
of  Non  Standard  Analysis  (  N£.A.)  .  A  macroscopic  observation  of  the  process  is  an 
observation  which  Is  not  able  to  distinguish  differences  at  the  microscopic  level  :  For  Instance 
if  the  position  of  the  process  at  time  t  is  x  we  consider  that  our  mesurement  is  not  able  to 
give  the  exact  value  of  x  but  merely  any  value  which  is  infinitely  close  to  x  ;  any  such  a  value 
is  idealized  by  the  shadow  of  x,  which  is  the  standard  real  number  infinitesimally  close  to 
X  .  Thus  the  property  ; 

"The  shadow  of  is  positive' 

is  a  sentence  which  makes  sense  in  the  language  of  N.S.A.  (but  not  in  the  conventional 
language  ant  thus  is  called  external  )  and  expresses  a  property  of  W.W.  sWiich  makes  no 
reference  to  dt .  It  does  not  depend  on  dt  provided  it  is  infinitesimal  .  By  this  way  we  recov^ 
universality  of  the  Idealization  . 

This  mathematical  model  in  which  the  lav  of  the  process  is  defined  at  the 
microscopic  level  (dt)  and  is  observed  at  a  mecroscopic  one  fits  very  well  with  the  Brownian 
motion  in  which  we  actually  observe  at  a  very  large  scale  (  say  1 0~^  for  position  and  1 0~^ 
sec  fdr  time  )  the  consequences  of  about  10^^'"! 0^^  kicks  per  second  by  the  molecules  of  a 
gaz  on  the  Brownian  particle  . 

In  this  approach  all  the  technicalities  associated  to  measure  theory  are 
suppressed  and  replaced  by  those  which  are  associated  to  the  use  of  the  richer  language  of 
N3.A.  Where  is  the  benefit  ?  The  benefit  is  in  the  fact  that  we  need  very  few  etements  of 
N5.A.  ,  without  comparison  with  what  we  need  from  mesure  theory  .  This  was  recognised  by 
E.  NELSON  [1  ]  .  In  this  book,  among  other  things,  he  establishes  in  less  than  80  pages  ( 
including  the  necessary  rudiments  of  NS.A.  )  the  extenW  equivalents  of  an  the  essential 
properties  end  caracterisations  of  the  Wiener  process  .  In  our  paper  [2]  we  have  extended  the 
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approach  of  NELSON  to  the  more  general  processes : 


“  5t h(Et.t)<«+»(5t)  2x^01 

t  €  {  0.  dt. 2dt,...j)dt . ndt  =  i} 


5o  =  0 


In  this  lecture  we  shall  explain  ; 

1 )  What  means  “almost  sure'  on  a  finite  set  with  uniform  probability  . 

2)  What  means  “( nearly )continuous'  for  a  discrete  mapping  from 

{  0,  dt,  2dt,...,pdt,....ndt  =  i }  to  R  and  why  it  is  a  macroscopic  concept  . 

3}  How  one  can  prove  the  following  : 

Theorein  :  Consider  the  process  defined  by ; 

5t+dt  =?t‘>(?t.^)«-^s(?t)Ztydt 

t  6  {  0,  dt.  2dt,...j)dt,...jidt  =  1} 


?o  =  0 


where  dt  is  an  infinitesimal.  Z(  is  a  sequence  of  independent  random  variables  taking  values 
in  ±1  with  equal  probabilities  and  the  mappings  b(.,.)  end  s(.)  ere  standard  continuously 
differentiable,  bounded  with  their  derlvetive  .  Then  “almost  every'  trajectory  is  “(nearly) 
continuous' . 


/ 
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ABSTRACT 


The  concept  of  local  controllability  Is  investigated  for  non-relativistic  quantum 
systems.  Sufficient  conditions  will  be  sought  such  that  the  solution  of  the 
controlled  Schrodlnger  equation  can  be  guided,  over  a  short  time  interval,  to  any 
chosen  point  In  a  suitably  prescribed  neighborhood  of  the  solution  in  the  absence  of 
control.  Evolution  equations  which  arc  linear  in  the  controls  but  nonlinear  in  the 
quantum  state  ^  are  considered.  Our  formulation  and  analysis  will  (for  the  most 
part)  run  parallel  to  those  of  Hermes. 


I .  INTRODUCTION 

In  recent  years,  there  has  been  a  growing  interest  in  the  system  theoretic 
problems  of  filtering  and  control  of  quantum  mechanical  systems.  Several  note¬ 
worthy  efforts  exists:  (i)  Torn.  Huang  and  Clark  [1)  and  van  der  Schaft  [2]  have 
explored  the  formal  basis  for  the  modelling  of  quantum  mechanical  control  systems. 
(11)  Clark.  Tarn  and  their  associates  [3-6]  have  obtained  results  on  quantum 
nondemolition  filtering  problem.  (ill)  Belavkin  (7)  has  investigated  the 
measurement  and  control  problem  in  quantum  dynamical  systems.  (iv)  Pierce.  Dahleh 
and  Rabitz  [8]  have  studied  the  optimal  control  problem  of  quantum  mechanical 
systems.  (v)  Butkovskly  and  collaborators  have  discussed  the  control  of  quantum 
objects  in  broad  terms  and  have  set  forth  general  conditions  for  controllability  of 
pure  quantum  states  [9-11]. 

To  the  authors'  knowledge  very  little  has  been  published  in  the  way  of 
mathematically  definitive  results  on  the  controllability  of  quantum  systems.  In  [12] 
the  authors  are  able  to  establish  a  series  of  global  controllability  conditions  for 
the  Schrodlnger  equation  which  is  linear  in  state  and  linear  in  the  external 
controls  by  extending  the  geometric  approach  as  implemented  by  Sussmann  and 
Jurdjevic  [13,14],  Krener  [15],  Brockett  [16],  Kunita  [17]  and  others. 
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In  Che  present  conCrlbuClon,  we  shall  consider  evolution  equations  which  are 
linear  in  controls  but  nonlinear  in  Che  quantum  state;  in  this  case  the  work  of 
Hemes  [18]  is  extended  Co  obtain  conditions  for  local  controllability  along  an 
unguided  reference  solution. 

II.  PROBLEM  FORMULATION  WITH  NONLINEAR  GENERATORS 


In  adapting  Hemea'  work  [18]  Co  our  anda,  it  is  convenient  to  chink  in  terms 
of  the  X  representation  [19].  Thus  Che  state  vector  feH  will  be  represented  by 

Che  wave  function  f(x)  6  LjCR”),  where  xsR”  stands  (ordinarily)  for  the  set  of 

spatial  coordinate  variables  associated  with  Che  quantum  system.  (More  generally, 

X  may  stand  for  any  complete  set  of  compatible  variables  [19]  built  from  the 
position  and  momentum  variables.  Spin  and  other  internal  degrees  of  freedom  can  be 
incorporated  by  essentially  trivial  modifications.)  Now,  let  us  define  a  class  of 
operators  H  in  W  which  are  supposed  to  be  skew-Hemlcian  (norm  preserving)  and 
time  independent  and  to  have,  in  the  x  representation,  the  mode  of  action 


(Hf)(x)  -  Hfl 


-  fl.l<<«X.l«><>'»---^,q«»X.q«> 


(X)),  (1) 


Here,  p,  q  are  some  integers,  Che  H.  (X  -  l,.,.,p;  ii  -  1 . q)  are  closed, 

*  M 

skew-Hermician  linear  operators  acting  in  H,  and  the  mappings  f^  C^-  are 
real  analytic.  (By  the  last  requirement  we  mean  that  f^  is  a  real  analytic 

function  of  its  argument  w,  this  argument  in  itself  being  generally  complex,  w  g 


Also,  in  expression  (1),  f^  interpreted  as  the  usual 


product  of  complex  functions.)  Throughout  the  current  section,  the  generators 
Hg,...,H^  entering  the  "controlled  Schrodinger  equation"  will  be  assumed  to  be  of 
this  more  general  form.  Thus,  while  Hg,...,H^  are  still  taken  skew-Hermitian,  they 

need  not  be  linear- -although  the  linear  case  is  certainly  included. 

We  shall  further  assume  that  a  unique  local  solution  exists  for  Che  initial 
value  problem 

^  ^  -  K "  ’^t-o  -  •  '2) 

posed  by  the  Schrodinger  equation  so  generalized,  the  admissible  controls  u^  now 

being  real,  analytic,  bounded  functions  of  c.  To  establish  that  this  is  a  viable 
assumption,  we  note  that  it  is  automatically  fulfilled  within  the  framework  of  (12). 
provided  d  belongs  Co  Che  analytic  domain  D^;  moreover,  in  Ref.  20  it  has  bean 

shown  Co  be  valid  for  a  certain  relevant  class  of  partial  differential  equations. 

On  the  other  hand  Che  formulation  of  general  conditiona  on  Hq  *  Zw^H^  for  the 


•xlstenca  of  a  unique  local  solution  of  (2)  awaits  further  mathematical 
developments. 

Our  next  Cask  is  Co  specify  the  Lie  bracket  appropriate  to  the  (generally) 
Inflnice-dlmenslonal ,  (generally)  nonlinear  control  problem  (2),  wherein  the 

k-0 . r  are  of  type  (1).  First,  we  appeal  to  the  chain  rule  to  define  a  sort 

of  derivative  operator,  OH,  corresponding  to  an  operator  H  of  that  type: 


((DH4) 


(x))f(x)  - 


where  f  e  ff  and  f ' (w)  is  the  derivative  of  f(w)  with  respect  to  its  argument. 
The  Lie  bracket  of  two  operators  H,  K  of  Che  indicated  class  is  Chen  specified  by 


([H,K10(x)  -  (H,Klfl^  -  KDHf)(x))(K«)(x)  -  t  (DKC)  (x)  )  (H?)  (x)  ,  (.U) 

Co  apply  V  (  e  H  and  V  x.  Again  we  shall  employ  the  notation  ad^^K  -  [H,KJ, 

adj^"*"^  K  -  (H,ad‘'K],  ^  -  1,  2,...;  also,  ad^K  -  K.  The  prescription  (4)  for  the  Lie 

product  is  obviously  consistent  with  chat  of  112],  for,  if  H  and  K  are  linear, 
1H,K)  -  HK  -  KH  as  in  (12)  . 

Remark  I .  The  above  definitions  and  specifications  are  tenable  even  if  H  ant. 
^  of  (1)  are  not  skew-Hermician  (or  even  if  skew-Hermiclclcy  is  not  a  meaningful 

concept).  As  is  well  known,  skew-Hermicicicy  of  the  generators  of  time  displacement 
is  an  indispensible  requirement  in  conventional  quantum  theory,  where  it  is 
necessary  for  the  probability  interpretation  of  On  the  other  hand,  there  are 

circumstances  in  which  one  may  be  led  to  drop  this  requirement,  namely,  (i)  in 

approximate  treatments  of  the  Schrodinger  equation  designed  to  yield  simple  pictures 
of  complicated  phenomena  Involving  many  degrees  of  freedom,  and  (ii)  in  radical 
revisions  of  conventional  quantum  theory  aimed  at  a  more  fundamental  description  of 
the  microscopic  world.  The  optical  model  of  nuclear  reactions,  (21)  wherein  a 
complex  potential  is  Introduced  to  simulate  the  effects  of  inelastic  processes,  is  a 
good  example  of  circumstance  (1),  while  the  hadronic  theory  proposed  by  Santilli 
(22)  suffices  to  illustrate  possibility  (11).  Obviously,  in  the  latter  context  new 
interpretations  as  well  as  a  new  formal  apparatus  (see,  e.g..  Ref.  23)  must 
accompany  the  enlarged  mathematical  framework 

Remark  2 .  The  marsage  of  this  comment  is  similar  to  that  of  Remark  1,  except 
that  the  subject  is  nonlinearity  of  the  generators  Hq . rather  than 

violation  of  their  skaw-Harmlticity .  Conventional  quantum  mechanics  is  necessarily 
a  linear  theory,  in  that  the  superposition  principle  is  an  essential  property. 
Specifically,  linearity  of  the  operators  Hg . is  required  to  maintain  this 

property.  But  again  one  might  agree,  either  (i)  in  the  framework  of  approximation 
methods,  or  (il)  in  fundamental  extensions  of  quantum  theory,  to  sacrifice 
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llnaii../.  The  Hartree-Fock  approxlnaclon  (19,21]  of  atonic  and  nuclear  physics 
furnishes  a  pronlnenc  exanple  of  a  nonlinear  approxlnaclon  to  the  conventional 
quantum  description.  On  Che  ocher  side  of  Che  coin,  nonlinear  quantum  theories  ac 
Che  flrsc-prlnclples  level  have  been  considered  by  a  number  of  authors;  for  example. 
Winner  [24]  has  suggested  chat  a  resolution  of  the  mysteries  associated  with 
"collapse  of  the  wave  packet*  might  be  sought  In  terms  of  such  a  theory.  [25] 

III.  GENERALIZED  DECOMPOSITION  THEOREM 


Consider  the  syscen  (2),  wherein  It  Is  assumed  chat  ^  c  D  -  f~]  don  H.  * 

k-0  “ 

null  sec.  Let  V^<d)  r  ^  denote  the  solution  (evaluated  at  tine  t)  aZ  the 
associated  reference  problem 

^  "t  -  Vt  •  "O  - 

This  problem  corresponds  to  free  evolution  of  the  quantum  system,  the  external 
controls  being  turned  off;  accordingly  -  V^(d)  will  be  referred  to  as  the 

homoeeneous  reference  solution.  Treating  rewritten  f.  as  an  arbitrary  element 
of  the  allowed  domain  D.  we  obtain  a  mapping  c  -•  V^(f),  which  In  general  defines  a 

nonlinear  operator.  (We  note  chat  In  the  special  case  that  the  generator  Hq  is 
linear,  V^(;'),  which  traces  an  integral  curve  of  the  vector  field  H^,  serves  to 
define  a  linear  evolution  operator  V^.  However,  in  the  nonlinear  setting  of  the 

present  analysis,  we  are  strictly  not  allowed  to  divorce  operator  from  operand, 
since  an  operator  of  class  (1)  generally  depends  on  the  point  of  H  at  which  it 
acts.)  The  differential  of  the  mapping  f  -  v^(f),  to  be  denoted  DV^(f),  is  also 

(generally)  a  nonlinear  operator.  One  may  loosely  interpret  DV^(f),  as  the 
derivative  of  the  object  V^(0.  a  state  vector,  with  respect  to  its  argument,  which 
is  again  a  state  vector.  By  will  mean  the  differential  of  the  (wave 

function)  -•  (wave  function)  map  f(x)  -  -» 

Definition  1.  A  complex- valued  function  g:  t  -•  g(t)  -  gj^(c)  -r  lg2(t)  is  said 

to  bo  complex  analytic  in  the  variable  t,  where  t  c  R^,  If  the  functions  gj^  and 
$2  fAAl  analytic  in  t. 

Theorem  1.  (Generalized  Decomposition  Theorem  (cf.  Refs.  18,26)).  Let  ;  be 
an  arbitrary  element  of  the  common  domain  D  of  the  operators  Hq,  ...,  H^,  and 

suppose  that  (1)  the  maps  t  ->  V^((’)|^  and  t  »'*  complex  analytic  in  t 

for  all  X  and  (11)  the  differential  DV^(f)  converges  in  the  strong  operator 

topology  to  the  ide,iclty  operator  Id,  as  t  -•  0*.  Than,  a  sufficient  condition  for 
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V^(Wj.(^))  to  provide  a  solution  of  the  controlled  dynamical  problem  (2),  is  that 


satisfy 


i- 

dt 


«t  -  Jo  ^  Ko  J,  «o  -  ^ 


(6) 


If  DV^(^)  is  one-to-one,  Che  state  condition  is  also  necessary. 

Proof.  A  necessary  and  sufficient  condition  for  V^(W^(^))  to  be  a  solution 

r 

of  (2).  given  chat  -  Wo(^)  -  is  HoV|.(Wj.(*) )  +  u^H^V^(U^(4S) ) 


(7) 


Since  by  definition  V^(f)  must  satisfy  the  differential  equation  3V^(f)/3t  - 
HQV^(f),  where  f  may  be  regarded  as  an  independent  variable  so  far  as  the  time 

derivative  is  concerned,  the  initial  terms  in  the  first  and  last  members  of  (7) 
cancel.  Thus  condition  (7)  may  be  distilled  to 


DV^(d)  ^  W^(«)  -  u,(t)H,]v^(W^(d))  . 

The  crucial  step  is  to  prove  that,  V  f  and  V  x, 

,lo  ^  K  k  '‘<">"<1”-  •  [j, 


>1,^ 


(8) 


(9) 


Once  property  (9)  is  established,  the  theorem  is  in  hand;  for  if  W^(d)  satisfies 
(6),  It  will  then  follow  from  the  sufficiency  of  (8)  that  V^(W^(d))  solves  problem 
(2). 

In  order  to  establish  (9),  we  examine  the  quantity 


g/t;H,)lx  -  DV^(r)  ^ 


t(f>lx- 


(10) 


With  f  an  element  of  the  allowed  domain,  the  maps  t  -*  and  t  - 

are  complex  analytic  by  our  hypotheses,  as  is  Che  map  t  -•  DV^(f)|^.  Consequently, 

the  right-hand  side  of  (10)  is  complex  analytic  in  t,  for  all  f  and  for  all  x. 
Therefore  it  is  legitimate  to  evaluate  means  of  its  Taylor  expansion 

in  t . 

To  begin  with,  we  know  g^(0;H^)|j^  -  0.  because  DV^(f)  -  id  in  the  strong 

operator  topology  as  t  -•  0^,  and  -  f.  Next,  consider  chat 

^DVtCO  -  v^(f)  -  D^(HoV^(0)  -  D^(Ho(V^(0)l 

-  (DHg(V^(f)))|{DVj.(C)l 

(The  differencials  in  the  first  line  are  all  with  respect  to  (.as  is  indicated 
explicitly  in  places  where  confusion  might  arise.  The  differencial  0^ (Hq(V^(C) ) ]  is 
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compuced  as  cha  produce  o£  cha  dlffaranclal  of  the  napping  V^(f)  -  Hp(V^(j-))  and 
Che  differencial  of  Che  inicial  napping  f  -•  V^(c}.)  In  similar  vein, 

-  DH_,(V^(r))HoVj.(f)  . 

Using  Chese  lasc  Cwo  relacions,  we  may  obtain  (with  Che  doc  indicating  cine 


derivative) 


g^(t:H^)  -  DHo(Vj.(r))DV^(r)  I  H^)r 

l/mtO  0 


i/-0  0 

-  DHg(V  (!-))[DV^(r)  i  tadj^  *  H^V^(r)] 

I/-0  0 

+  (DHg(V^(f))H^V|.(f)  -  DH^(Vj.(f))HQV^(r)) 

-  DHQ(V^(0)g^(t;H^)  -  g^(t;adj,^H^)  . 

But  we  know,  from  previous  argument  or  its  extension,  chat  and 


(11) 


g^(t;adjj  tend  to  zero  as  t  -♦  0  ;  it  follows  that  ®  ^ 

and  for  all  x. 

The  pattern  is  now  sec  for  an  inductive  construction  of  successive  time 
derivatives  of  g(c;H^).  In  particular,  based  on  the  above  results  we  may  form 

^  [DHQ(V^^'>)g^(t;H^)l  +  DHQ(Vj.(f))g^(t;Hp 

■  DHo(V^(r))g,(c;adj,^Hp  .  g,(t;ad^^Hp  , 


and  it  follows  chat  "*  0  a® 


Continuing  the  process  indefinitely. 


we  arrive  at  the  result  that  at  t  -  0  all  the  time  derivatives  of 

vanish,  to  arbitrarily  high  order.  Thus  identically  0,  Vf ,  Vx.  i.e., 

i/— 0  0 

f  -  1 . r.  The  desired  property  (9)  ensues  upon  multiplying  this  equality  by 

u^(t)  and  summing  over  i. 

Corollary  1.  Same  as  Theorem  1.  except  that  "complex  analytic"  is  everywhere 
to  be  replaced  by  "re.il  analytic".  (See  Ref.  IB) 

Proof.  Direct  observation. 


199 


IV.  LOCAL  CONTROLIABILITY  ALONG  A  REFERENCE  HOMOGENEOUS  SOLOTIOW 

Definlelon  2.  The  system  (2)  is  said  to  be  locally  controllable  along  the 
solution  7^  -  V^(^)  of  the  control-free  problem  (S)  on  the  manifold  H  c  H  if,  for 

small  t  >  0,  there  exists  a  sec  of  Uj(c),  i  -  1 . r,  such  chat  the  solution 

of  (2)  can  be  controlled  to  a  neighborhood  of  7^  on  H.  The  precise  meaning  of 
Che  last  phrase  is  Chat  can  be  steered  into  any  direction  of  Che  tangent  space 

of  M  at  Che  point  7^  -  V^(^)  c  H,  V  4  t  H. 

We  may  now  formulate  the  central  result  of  this  section. 

Theorem  2 .  Assume  that  Che  homogeneous  solution  of  system  (2),  i.e.,  the 
solution  7^  -  V^(^)  of  Che  uncontrolled  system  (5),  satisfies  the  hypotheses  (1) 

and  (11)  of  Theorem  1  for  C  (and  specifically  on  a  finite-dimensional 
submanifold  H,  M  c  0  C  N,  dim  H  -  m.  Assume  further  that  there  exist  integers 

(with  J  -  1 . r  and  -  1 . °  -  ‘'il  ‘'£2  ^  <  •'jv  ^ 


■‘Jl 

such  that  the  set  ( [adu  H.]^)  spans  the  tangent  space  7M  of  M  at  7  -  V  (^) 
TIq  i  ’’t  ' 

for  all  ^  c  M.  It  follows  that  system  (2)  is  locally  controllable  along  7^  on  M. 

(Cf.  Theorem  2,  Ref.  18.) 

Proof .  If  the  functions  u^j  (c)  ,  where  1-1 . r  and  -  1 . k^, 

qualify  as  admissible  controls  (real,  analytic,  bounded  functions  of  t) ,  Chen  so  do 
Che  finite  linear  combinations 
k. 


u/(t)  X  a  u 

j  .-1  •'Jf 


(t). 


1-1. 


wherein  the  real  coefficients  a 


11  • 


a^j^  ,  are  chosen  (for  convenience)  to  obey 


Jl-l 


,1-1 . r  . 


Let  us  abbreviate  Che  sec  (a,,  )  simply  as  g.  By  generalized  decomposition  in  the 

‘'Jl 

multi-input,  complex  case  of  Che  preceding  subsection  (i.e.,  by  virtue  of  Theorem  1, 


Che 


solution  of  problem  (2),  with  the  u^  as  controls,  is  given  by 


0®  -  V^(W®(0)).  The  solution  f®  -  W®(0)  of  the  boundary  value  problem  (6), 

“i 

restated  for  the  controls  u^  ,  evidently  obeys  the  integral  equation 

w*(0)  -  ^  ^  “?<t)  ^  ds  [.<i;;^hJv^(0)  . 


Thus 


V^(W^(«))  I..0  -  “V,(W-(^))  W^(^)  |._o 

Ji 

-  Jo  [/S  "ij/*)  ^  «^]'>\<^>[So  »i]^  • 

where  a-  0  means  all  of  the  a,,  are  zero.  By  assumption,  we  can  find  a  sec  of 
integral  (or  zero)  powers  k^j  ,  where  f-1.  ...,r,  j^-1 . k^, 

0  <  i/^j^  <  k^2  <  •  •  •  <  ‘'jjjj  .  ‘'nax  ”  j  ' 

‘'ll  ‘'^h 

([ad„  H.]^ . (adu  H,]  i-1 . r)  spans  TH  Then,  since  (also  by 

TIq  i  Hq  ■*  It 

assumption)  DV^(^)  -•  Id  strongly  as  C  -►  0^,  there  must  exist  a  time  tj^  >  0  such  that 

the  set 

. »i]4 

spans  TM  ,  over  the  time  Interval  0  <  t  <  t, . 

’>t  ^ 

Ve  now  proceed  Co  make  a  Judicious  choice  of  the  original  functions  u..  (c) 


involved  in  (12).  One  can  realize  admissible  controls  u..  (t)  obeying  the 


conditions 


/o 


0,  for  vp-v,,  ,  0  <  V  <  V  +  1  , 

zj ,  -  max 


c,.  w  0  .  for  -  u, 


where  f-1 . r.  j^-1 . k^.  and  the  c^j  are  real  constants.  The 

connection  between  the  u,,  and  Che  u.  will  be  specified  shortly.  The  power  v 

being  integral,  inversion  of  relations  (13)  is  in  effect  Ju't  a  classical  finite- 
moments  problem.  (Note  that  in  the  upper  range  v  >  +  1,  we  have 


t,  /  /■  w  +  3\ 


since  |u,,  I  is  by  assumption  bounded.  This  implies  chat  the  higher  moments  not 

specified  by  (13)  will  be  negligible.) 

With  t  in  the  interval  [0,Cj^J,  we  now  carry  out’the  change  of  variable 

s  -  tj^h/t  In  Che  integral  on  the  left  of  (13) : 


0,  for  V  r‘  V,,  ,  0  <  V  <  V  +1. 

*  —  mav  * 


>  1/  +1 
max 


^  -  \  (^)‘"'^  .  for  .  -  . 

^  oft*'"-*  ■"  V  for  .  >  .  +  1  . 

1.  J  ’  max 

Setting  u,.  (s)  -  U(,  (t.s/t)  in  (12).  we  arrive  finally  at  the  result 
■'Ji 

3^  V^(wt(^))  l,_o  -  DV^(^>  ^  0  p-  "  ')  • 

where,  for  t  <  t^,  the  last  tern  can  be  neglected,  t^  being  small.  Consequently  the 
set  (aV^(W^(^))/aa^j  ,  i-1.  ...,  r,  j^“l . spans  TM^  for  t  in  the 

interval  (0,t^],  where  t^  has  been  chosen  above.  This  means  that  we  have  been 

able  to  choose  the  controls  so  that,  for  small  t  >  0,  the  state  defined  by  system 
(2)  can  be  steered  into  any  direction  of  the  tangent  space  on  M  at  the  point  - 

V^(^) .  Then  by  definition  the  system  is  locally  controllable  along  the  reference 
solution  V^(^) ,  for  all  ^  c  H. 

Remark  3.  Theorems  1  and  2  remain  true  as  stated  if  the  Hj^.  k  -  0 . r,  are 

not  skew-Hermitlan. 

Example  1.  The  theorems  of  the  present  paper  are  aimed  at  an  infinite- 
dimensional  space  of  quantum  states.  However,  the  results  obtained  herein  are  still 
valid  (with  trivial  alterations)  for  a  finite-dimensional  state  space.  As  pointed 
out  in  Remark  3,  from  a  mathematical  standpoint  we  may  also  dispense  with  the 
assumption  that  the  generators  Hq,  ....  are  skew-Hermitian. 

For  example,  consider  a  nonlinear  control  system  on  r"*,  m  <  •>,  defined  by 

^  X(C)  -  A(x(t))  +  u(t)B(x<t))  ,  x(0)  -  Xq  . 

where  A  and  B  are  real  analytic  vector  fields  corresponding  to  nonlinear 
operators  of  the  sort  introduced  in  Section  II.  Then,  as  argued  in  Ref.  IS,  a 

sufficient  condition  for  local  controllability  along  the  homogeneous  (u  >  0) 
solution  of  (14)  is  rank{  [ad^BJxQ,  -  0,1,2 . •)  -  m.  This  is  precisely  the 

condition  which  would  enter  the  finite-state-space  version  of  Theorem  2.  Problem 
(14)  does  not  strictly  refer  to  a  quantum-mechanical  system;  its  study  is, 
nevertheless.  Illuminating. 

While  surely  of  high  interest,  the  identification  and  analysis  of  "non- trivial" 
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•xanples  of  th«  utility  of  Theoren  2,  naaning  •xamples  concornod  with  novel  quantun 
control  systems  characterized  by  nonlinear  genarators,  exceeds  the  scope  of  the 
present  work. 

V.  SUMMARY  AND  OUTLOOK 

It  has  been  our  aim  to  augment  the  foundation  for  the  concept  of 
controllability  of  quantum-mechanical  systems  [12].  In  the  generalited,  nonlinear 
formulation  of  the  quantum  control  problem,  we  were  able  to  determine  conditions  for 
the  property  of  local  controllability  along  a  homogeneous  (i.e.,  control-free) 
solution,  without  having  to  refer  to  the  existence  of  an  analytic  domain  which  was 
assumed  in  the  global  analysis  of  [12].  (Our  treatment  of  this  case  amounts  to  an 
extension  of  Hermes'  work  [18]  to  a  multi-input,  complex- state  problem.)  From  the 
results  obtained  herein  on  the  controllability  of  the  solution  of  nonlinear 
Schrodinger  equations,  one  may  regain,  upon  appropriate  specialization  or 
adaptation,  certain  well-known  systems -theoretic  results  in  finite-dimensional  state 
space  (see,  in  particular.  Refs.  13-18). 

Clearly,  only  a  modest  beginning  has  been  made  toward  achieving  Che  larger  goal 
of  a  comprehensive  theory  of  quantum  control.  The  following  problems,  among  others, 
await  concerted  effort: 

(1)  Adaptation  of  the  notions  of  observability,  Identification,  realization, 
and  feedback  to  the  quantum  context  [27]. 

(ii)  Study  of  a  controlled  version  of  the  Schrodinger  equation  for  the  time 
evolution  of  the  density  operator,  [19]  so  as  Co  extend  control  theory  to  the  realm 
of  quantum  statistical  mechanics. 

It  is  evident  chat  powerful  mathematical  techniques  must  be  Invoked  Co  carry 
through  this  program;  moreover,  one  must  confront  the  profound  conceptual  obstacles 
intrinsic  to  the  quantum  measurement  process  [25,28,29]. 
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1 .  Introduction 

In  recent  years,  interest  has  developed  in  the  connections  between  stochastic 
control  theory,  dynamic  programming  and  quantum  mechanics  [1-A,  7,  12,  13]  and  (related) 
variational  approaches  [9,  11,  14,  15]  to  Nelson's  stochastic  mechanics  [10].  In  this 
paper,  we  will  start  by  considering  two  stochastic  optimal  control  problems,  one 
"forward"  in  time,  one  "backward"  in  time.  We  show  that,  if  there  are  solutions  to  the 
extended  Hamilton- Jacobi  equations  associated  with  the  control  problems,  then  there  is  a 
solution  of  a  Schrodinger  equation  and  conversely,  if  there  is  a  sufficiently 
well-behaved  solution  to  a  Schrodinger  equation,  there  are  solutions  to  a  pair  of 
extended  H-J  equations.  We  note  the  connection  between  the  H-J  equations  and  the  main 
dynamical  equations  of  Nelson's  stochastic  mechanics.  The  H-J  equations  are  equivalent 
to  a  pair  of  inhomogeneous  "backward"  and  "forward"  heat  equations  via  a  well-known 
exponential  transformation.  One  may  thus  pass  from  these  to  a  Schrodinger  equation  (and 
back) . 

2 .  Definitions  and  Notations 

We  assume  a  given  underlying  probability  space  (n,F,P).  E*^  denotes  n-dimensional 

Euclidean  space,  (t(,,t,)  an  interval  in  E’ .  S  denotes  (t^.t,)  x  e";  S  -  [to.t,]  x  e". 
Definitions  of  stochastic  process.  Brownian  motion  will  be  taken  from  |6)  as  will  other 
elements  of  our  framework  which  will  be  noted  below. 

A  solution  of  a  stochastic  differential  equations 

dl  -  b(t,?(t))dt  +  o(t,5(t))dw  (2.1) 

with  initial  data  f(s)  -  y  is  to  be  interpreted  as  in  [6]  as  a  solution  of  the  integral 
equation 

f(t)  -  C(s)  +  J  b(r,f(r))dr  +  |  a(r,€(r))dw(r)  (2.2) 

s  s 

Here,  w  is  standard  Brownian  motion  of  dimension  n.  With  the  vector  notation 
■ -Cn) .  b-(b, ...bn),  we  have 

n 

•*^1  “  bi(t,f(t))dt  +  J]  o£_j(t,<(t))dw,{  1-1  ,.  .  .n 

f-1 


206 


The  notation  C^'^(S)  denotes  the  class  o£  functions  0  In  '^(S)  (oeanlng  c’  In  t, 
C^ln  X)  which  satisfy  |^(t,x)|  s  D(l+|x|^)  for  soae  constants  D,k,  when  (t,x)  t  S. 

3 .  Two  Stochastic  Optimal  Control  Probleas 

We  consider  first  a  "forward"  stochastic  optimal  control  problem,  Problem  1,  in  3.1, 
then  the  symmetric  "backward"  problem.  Problem  2,  in  3.2.  The  controls  v  and  V  will  take 
values  in  e". 

3 . 1  Problem  1 

Consider  the  stochastic  differential  equation 

df  -  v<t,4(t))dt  +  odw  (3.1.1) 

with  initial  data  ^(s)  -  x  <  e",  at  time  s  «  (to,t, ).  Here,  w  is  a  standard  n-dimensional 
Brownian  motion,  and 

"ij  ■  ^ij 

where  f  is  the  Kronecker  delta,  and  D  is  a  positive  constant.  We  assume  that  v  belongs 
to  a  class  of  admissible  control  functions  defined  as  follows: 


Definition  3.1. A  [61.  A  feedback  control  law  v  (the  term  feedback  refers  to  the  fact 
chat  the  control  is  a  function  of  the  state  f(t))  is  admissible  if  v  is  a  Borel 
measurable  function  from  S  into  e",  such  that 

(a)  For  each  (s,x),  Cq  <  s  S  t^  ,  there  exists  a  Brownian  motion  w  such  that  (3.1.1) 
with  initial  data  ^(s)  -  x  has  a  solution  f,  unique  in  probability  law 

(b)  For  each  k  >  0,  Esxlf(t>l''  is  bounded  for  s  s  t  S  t^  ,  and 

^sx  J  |v(t,f(t))l''  dt  <  «. 

s 

(the  bound  may  depend  on  (s,x)).  The  subscript  sx  refers  to  Che  fact  that 
f(s)  -  X. 

Either  of  Che  following  conditions  are  sufficient  for  the  admissibility  of  v: 

(i)  For  some  constant  ,  |v(t,y)l  S  M^(l+|y])  for  all  (c,y)  e  S.  Moreover,  for  any 
bounded  Borel  set  B  c  e"  and  Cg  <  c'  <  Cf ,  there  exists  a  constant  such  that,  for  all 
x,y  t  B  and  tg  S  t  s  t'  , 

lv(c,x)  -  v(t,y)|  s  K, |x-y| 


(Kf  may  depend  on  B,c';  and  both  ,  K,  siay  depend  on  v) . 
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(ii)  V  satisfies  a  Upschitz  condition  on  S.  Further,  if  (i)  or  (ii)  holds,  the 
Brownian  notion  w  can  be  specified  in  advance,  which  is  the  case  in  ProblsB  1. 

Now,  for  (t,x)  e  S  and  u  t  e",  let 

L(t,x,v)  ■  j  au^  +  Q(t,x) 

where  Q  is  continuous  on  S,  and  let 
be  continuous  and  assume 

|Q(c,x)|  s  C(1  +  |x|)'‘ 
tf,(x)  s  C(1  +  |x|)'‘ 

for  some  constants  C,k. 

We  define  a  cost  function 

J(s,x.v)  .  Ejx  11*^’  L(t.f(t),v(t.^(c))  dt  +  W,  ({(t,)]-  . 

s 

The  conditions  on  Q  and  W^  ensure  that  J  is  finite. 

Now  let  the  optimal  control  problem  be  as  follows;  Find  an  admissible  feedback 
control  v*.  among  all  admissible  feedback  controls,  which  minimizes  J(s,x,v).  The 
following  Verification  Theorem  gives  sufficient  conditions  for  the  existence  of  a 
minimizing  v*. 


(3.1.2) 

:  E°  (R.).  denoting  non-negative  real  numbers) 

(3.1.3) 


Theorem  3 . 1 .B  f6 1 .  Let  W(s,x)  be  a  solution  of  the  dynamic  programming  equation 

(3.1.4) 


(3.1.5) 

such  that  U  is  in  Cp'^(S)  and  continuous  on  S.  Then, 

(a)  W(s,x)  i  J(s,x,v)  for  any  admissible  feedback  control  v  and  any  initial  data 
(s,x)  €  S. 

(b)  If  V*  is  an  admissible  feedback  control  such  that 


DOW  +  J^vf(s.x)  +  I  m(v*<s,x))2  +  Q(s,x) 

-  [  DdW  +  ^  +  i  mu2  *  Q(.,x)]  (3.1.4) 

for  all  (s.x)  <  S,  then  W(s,x)  -  J(s,x.v*)  for  all  (s,x)  i  S. 


TV...e. 


^4  ^  ^ 
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Mow  lac  us  assuoa  chac  Chaca  axtscs  a  V  satisfying  Cha  hypochasaa  of  cho 
Varlflcatlon  Thaocaa,  and  an  optlaal  conccol  v*.  Than,  slnca  cha  eoncrols  taka  valuas 
In  e",  which  Is  open 

isv*  -  -  grad  W  for  all  (s,x)  «  S  (3.1.6) 

and 

-  -  DOW  +  ^  (grad  W)*  -  Q  (3.1.7) 

for  all  (s,x)  <  S.  Equation  (3.1.7)  Is  analogous  to  the  Hanilcon- Jacobi  equation  of 
classical  aechanies;  wa  shall  refer  to  it  as  an  extended  Hanilcon- Jacobi  equation. 


3.2  Problem  2 

Now  let  us  introduce  another  type  of  adnissibilicy  for  a  feedback  control  function 
as  follows: 

Definition  3. 2 .A  A  feedback  control  law  v  is  backward  admissible  if  v  is  such  that 

v(r,x)  -  -  v(to+ti-r,x)  for  all  (r,x)  <  S,  and 
V  is  an  admissible  feedback  control  law. 


Ue  consider  the  stochastic  differential  equation 

di7  “  v(r.i)(r))  dr  +  J2D  dw  (3.2.1) 

where  v  is  a  backward  admissible  feedback  control  law,  and 
«(*■)  -  w(Co+t,  -r) . 

We  say  Chat  ij  is  a  solution  to  (3,2.1)  with  terminal  data  r((o)  -ye  e",  with 
Cfl  s  r  <  o  s  c,  ,  If  rj  satisfies  the  integral  equation 


fj(r)  -  n(o)  -  j^V(r  .i)(r))  dr  -  1^/25  <W(r). 


(3.2,2) 


By  making  the  change  of  variable 


r  -  to  +  C,  -  t, 

<7  -  to  +  t,  -  s, 

J  -  Co  +  c,  -  r 

(3.2.2)  becomes 

n(to+t, -t)  -  q(to+ti-s)  -  I  V(t,+t, -I,  q(t,+t, -I))  (-di) 


r.)  - 
- 1*  yso  <»  (to+t,.f) 


(3.2.3) 
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Dafln*  i»(c)  ■  ir(to+Ci-t).  Now  (3.2.3)  boeoMs 

n(t)  -  n(s)  +  J  V  (i.i»(i))  di  +  J  755  <»(i)  . 

i  • 

«nd  we  have 

^(s)  -  y 

We  now  define 


(3.2.4) 


(3.2.5) 


J(s.y.v)  ■  E, 


-  E, 


vy 


I  !  Q(i,i(i))ldi  +  Uo(i(t,))| 

I  |^[f  (v(r.,(r))]*  +  Q(r,i,(r))Jdr  +  Wo  (i?(to))| 

-  J(ff.y.^)  . 


(3.2.6) 


Here  Q  is  the  sane  as  In  Problen  1,  W^ ;  (L).  Is  continuous  and 
Wo(y)  s  C(1  +  |y|)'',  (C.k  as  in  (3.1.3)),  Thus,  <3(1. i(f))  -  Q(Co+C, -1,  fT(to+t, -1) ) . 

We  now  consider,  as  In  Problem  1,  Che  problen  of  ninimlzlng  (3.2.6).  For  given 
Cemlnal  data  y  <  e"  at  time  o  <  (to.C,  ],  we  shall  say  that  is  backward  opcinal  if 
7^  is  backward  admissible,  and 


J  (ff.y,<?)  i  J  (o,y,7^) 
for  all  backward  admissible  7. 

In  view  of  (3.2.4)  ■  (3.2.6),  we  have  the  following  version  of  Che  Verification 
Theorem: 

Theorem  3.2.B  Let  W  be  a  solution  of  the  dynamic  programming  equation 


0-^+5'" 

°  as  u«E" 


[  OdW  +  I  +  Q(».y)] 


(3.2.7) 


(s.y)  «  S 


with  W(tj  ,y)  -  Wo(y),  y  «  e",  such  that  W  is  in  cJ,'*(S)  and  continuous  on  S.  Then: 


(a)  W(s,y)  £  J  (s,y,v)  for  any  admissible  feedback  control  v  and  any  initial  data 
(s.y)  (  S. 

(b)  If  V*  is  an  admissible  feedback  control  such  chat 


Daw  +  vf  (s.y)  +  I  n(>(s.y))*  +  Q(s.y)  - 


[  DdW  J  *  Q<».y)] 


ucE 


(3.2.8) 


/ 
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*  ^  A  A  . 

for  all  (s.y)  €  S,  then  U(a,y)  -  J(s.y.v*)  for  all  (a.y)  «  S;  v  la  optimal 
Now  suppose  there  exists  a  function  W  satisfying  these  hypotheses, 
and  an  optimal  control  v*.  Define 

W(c.y)  ■  W<to+t, -<r,y)  ,  tj  <  ff  S  tj  . 

—  ^  /lU 

Then  W(to.y)  -  W(t,y)  and  ^  ~  ~  ■ 

Ue  define 

v*(c.y)  -  v^(to+=l -s.y)  -  -  V*  <s,y). 

Nov  we  have 

0  -  -  1^  +  Dow  -  (v^i(o.y))  1^  +  I  m(v^(o,y))*  +  Q(o,y)  (3.2.9) 

and.  as  in  Problem  1, 

mv^  -  grad  W  (3.2.10) 

|H  .  DOW  -  ^  (grad  W)^  ♦  Q  on  S.  (3.2 

Ue  have  let 

v*(s,y)  -  -  v^(Co  +  c, -s,y)  .  y  «  e". 

From  (3.2.6)  we  have 

■  (3.2.12) 

If  Che  Verification  Theorem  3.2.B  la  satisfied,  then  V*  Is  optimal;  that  Is 

J(s.y,v)  6  j(s,y,v*) .  ,3.2.13; 

From  (3.2.6)  and  (3.2.11),  (3.2.12)  Implies 

J(<’.y,'^)  S  J  (o,y,v^).  (3.2.14) 

for  all  backward  admissible  v. 

Therefore,  if  v*  Is  an  optimal  control  in  the  sense  of  Theorem  3. 2. A.  then  is  a 
backward  optimal  control  for  Problem  2,  and  the  ccn-rerjc  Is  tlso  true. 
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It.  Extended  Hamilton -Jacobi  Eguatlona.  the  SehrodlnKT  Equation 
end  Inhonogeneoua  Beckwerd  end  Fotvetd  Meet  Eouetlone. 

4.1  Extended  Hanlltcn-Jecobl  Equetlons  end  the  Schrodtnger  Equation 

We  have  seen,  that  If  there  exist  U.  W,  v*.  W*  satisfying  the  conditions 
of  the  Verification  Theorems,  Chen  W  is  a  solution  of  the  aquation 

(4.1.1) 

(4.1.2) 

and  W  is  a  solution  of  Che  equation 


It  "  is  G(t.x))  +  D4G(t,x)  +  Q(c.x)  - 

(t.x)  «  S 


with 


G(Ci  .x)  -  W,  (x). 


ft  is  G(t,x))^  -  D4G(c,x)  -  Q(t,x)  -  0 

(t.x)  «  S 


(4.1.3) 


with 


G(to.x)  -  Wo<x) 


(4.1.4) 


We  now  show  that,  when  there  are  solutions  G.  G  of  (4.1.1),  (4.1.3),  then  there  are 
solutions  of  a  Schrodinger  equation.  From  now  on  D  shall  denote  */2m. 

Proceeding  as  in  (4] ,  with  G*  -  ,  h*  -  ,  we  have 


(G*-H*)  -  (grad(G*-H*))*  +  D4(G*-H*)  +  Q  -  0 


(G*+H*)  +  ^  (grad(G*+H*))  +  04(G*+H*)  -  Q  -  0 


(4.1.5) 

(4.1.6) 


Adding  and  subtracting  (4.1.5),  (4.1.6)  gives 

It*  is  *  is  ^8"**  °  (*  l-7) 

♦  -  grad  H*  grad  G*  -  DoH*  -  0  (4.1.8) 

0  C  B 


Equations  (4.1.7),  (4.1.8)  are  equations  (19),  (20),  of  (4),  except  for  the  potential 
Q  which  was  taken  to  be  zero  in  [4J . 

At  this  stsge,  we  make  the  following  observation:  if  we  define 

5  ■  i  (VC*)*  -  2D4C*  -  Q 


(4.1.9) 
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then  (4.1.7)  becomes 


+  I;;  (grad  H*)*  -  ^  (grad  G*)*  +  DdG*  +  Q  -  0 


dc  2m 


(4.1.8)  is  unchanged: 


i  grad  H*  grad  G*  -  DdH*  -  0 

dC  ffl 


\ 


(4.1.10) 


(4.1.8) 


If  we  now  multiply  (4.1.10)  by  i,  and  stibtract  (4.1.8),  we  obtain 
|-  (-G*+iH*)  -  -  DaH*  +  -  grad  H*  grad  G*  +  |-  (grad  G*)^ 


2m 


^  (grad  H*)^  -  iDAG*  -  i  Q 


1^  (-G*+iH*)  -  iDA(-G*+iH*)  +  ^  (grad( -G*+iH*) )^  -iQ 


Straightforward  differentiation  gives  us 

Proposition  4 . 1 . A.  If  G,  G  are  solutions  to  (4.1.1),  (4.1.3),  then 


■  exp 


(^) 


.1  a  solut..r  to 

8 1  2m 


,  fC7G*)^  4AG*  ^ 

+  [— - ^  -  QJ0. 


(4.1,11) 


(4.1  l?i 


(4,l.lsj 


Conversely,  suppose  we  start  with  the  Schrodinger  equation 


ih  dri>  -fi 


-  PV»  (t.x)  <  S 


(4.1.:  .. 


dt  2m 

with  given  potential  P.  Assume  there  is  a  solution  0  of  (4.1.14),  0  0,  all  (t.x>,  wit.. 

f-M+iNl 


}/>  •  exp 


«  j 


(4.1. IS) 


and  suppose  that  M  and  N  are  c' functions  on  S.  Running  the  above  arguments  backwards, 
we  see 


1^  +  ^  (grad  N)^  -  (grad  M)^  DaM  -  F  -  0 

+  -  grad  N  grad  M  -  OaN  -  0 
d  t  m 

The  passage  cro.:i  '4.  ;.u  »  -o  ‘.Mto  pair  o:  i 


(4.1. 

(4.1.17.) 


eC  -OUIS 
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de  Broglie  for  Incroducing  his  ‘chaorie  du  guldaga*  (aaa  [5];  aquations  (&.1.16), 

(4.1.17)  are  the  so-called  equations  (J)  and  (C)  of  Louis  da  Broglia) .  Togethar  with 
this  pair  of  equations  he  defined  Che  quanCua  potential  <)p  by 

Qp  -  DAM  -  (grad  M)*  (4.1.18) 

The  purpose  of  the  definition  (4.1.18)  was  to  reduce  equation  (4.1.16)  to  Che  fora 

If  N)*  +  Qp  -  P  -  0  (4.1.19) 

which  is  the  Hamilton-Jacobi  eqxiatloti  of  classical  aechanics  for  the  notion  of  a  nass- 
point  in  the  potential  P  -  Qp.  As  the  reader  nay  anticipate.  If  we  next 
introduce  the  "modified  potential"  Q  by 

A  2 

Q  -  P  -  2Q„  -  P  -  2D4M  +•  W) 

^  m  ’ 

then 

f^CN+M)  +  ~  (grad  (N+M))^  -  Dc(N+M)  -  Q  -  0 

Ij(M-N)  -  ^  (grad  (M-N))^  +  Dc(M-N)  +  Q  -  0 

Thus  we  have 
Proposition  4.I.B.  If 

1  f-H+lNl 

^  -  exp 

is  a  solution  as  above  to 

H  ' 

then  (M-N)  is  a  solution  of 

BG  1  2 

at  ~  liS  (8rad  C)  +  DnG  +  q  -  0 

and  (N+M)  is  a  solution  of 

ac  1  -  2  _ 

at  25  -  Dag  -  Q  -  0  (4.1.24) 

Equations  (4.1.23,  (4.1.24)  are  the  equations  (4,1.1),  (4.1.3)  with  Q  raplaead  by  Q, 
which  is  given  by  (4.1.20)  (note  that  Q  is  specif lad  ones  P  is  given,  end 


(4,2.12) 


(4.1.23) 


(4.1.20) 

(4.1.21) 

(4.1.22) 


\ 


Remark  4.1.C 
Nelson's  Equations 

If  we  take  gradients  of  equations  (4.1.8),  (4.1.10)  and  define 


we  obtain 


3u  A  .  gred  grad  _  ^9 

at  "  2m  2  “  2  m 


(4.1.25) 


_  .  _  .  V(o-u) 


(4.1.26) 


Nelson  derived  these  equations,  which  are  the  main  dynamical  equations  in  his  theory 
theory,  via  different  methods  (Q/m  representing  the  force  field  acting  on  a  microscopic 
microscopic  particle  undergoing  a  Brownian  motion).  He  found,  with 


u  -  -  7R 
m 


u  -  -  vs 
m 


0  -  exp(R+iS) 


satisfied 

and  the  converse. 

Doing  this  involved  recognizing  that  (4.1.10),  (4.1.8)  (equivalently  (4.1.17), 
^4.1.16)  or  C  and  J  of  Louis  de  Broglie)  are  the  Imaginary  anu  real  part  of  the 
Schrodinger  equation  (modulo  the  factor  of  J)  which  we  used  in  going  from  (4.1.10), 
(4.1.8)  to  Prop.  4.1. A. 

4 . 2  Inhomogeneous  "backward  and  forward"  heat  equations 
Now,  if  we  make  the  exponential  transformation 


d(t,x)  -  exp(-G(t,x)/A) 


(4.2.1) 


in  equation  (4.1.1),  we  have 


(4. 2. 2)1 


d(t,  ,x)  -  exp 


Similarly,  if 


d(t,x)  ■  exp 


(^1 


(42.3) 
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(4.2.12) 
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^  and  ara  non-nagaclve ,  continuous,  a  d  boundad  functions  on  e".  (Ue  rafat  to 
[8]  for  exlstance  theory.)  It  may  be  seen,  [8],  that 

^(t,x)  >  0,  and  9(t,x)  >0  In  S 


provided  that  neither  do  nor  di  vanishes  Identically.  Now,  defining  V,  W  by 

*  -  ('2s] 


d  -  exp 


(si 


we  see  thac  W  Is  a  solution  of 

3G  1  9 

It  ■  2m  ^S^ad  G)  +  DOG  +  R  -  0 

with 

G(C, ,x)  -  -8  log  di 
and  W  is  a  solution  of 

It  5m  ^8“^  -  DOG  -  R  -  0 

with 

G(tfl .x)  -  -ft  log  di 


(4.2.13) 


(4.2.14) 


(4.2.15) 


(4.2.16) 


(4.2.17) 


(4.2.16) 


Thus,  from  Proposition  4.1. A  and  the  above  arguments,  we  have 

Fact  II.  If  d.  d  are  solutions  to  the  Cauchy  problems  (4.2.9),  (4.2.11)  and  (4.2.10), 
(4.2.12),  then 


d  -  exp  [ 


where 


3*  _  HlH  n*  -  W-W 
2  •  ■  2 


satisfies 


Ift 


§5  _  zsi  Ad  +  K(W+8)/2)]*-  - 

3t  2m  -  '  ^2m  J 


(4.2.19) 


..a .. 
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t 

I 


gote: 

•)  the  solution  ?  of  the  Sehrodlnger  equation  (4.2.19)  depends,  like  W  and  3,  on 
the  Initial  and  ternlnal  data  of  the  Cauct^  prohleiis. 

b)  -  _ 

||?(t.x)||*  -  exp  W(t,x?j  . 

“  ?*(t.x)  .  (4.2.20) 


Fact  I  Is  obtained  In  the  proof  of  Theorea  4.3  of  (15):  Fact  H  Is  nore  or  less  Inpllcit 
In  Theorem  4.4  and  Corollary  4.4.1  of  (15),  however,  the  arguments  here  give  Fact  II 
more  directly. 


Example  Homogeneous  "backward  and  forward**  heat  equations,  n  *  1 
The  solution  of 
di 

.  -Dad  on  (Cq  ,ti  ]  where  0  <  Cq  <  <  T, 


(sofe)] 


Is  known  to  be 


‘"p  1^5^)  'o  ^ 


(4.2.21) 

(4.2.22) 


(4.2.23) 


Similarly,  the  solution  of 


§i  ,  n 

at  “  at^ 

Is  known  to  be 

>o(b.*)  ”  7==  ‘xp  fz^l  • 
l^DtJ 

Then 

3(t,x)  -  -2nD  log  J(t.x) 
m  x^ 

“  J  ^  +  nDlog(T-t)  +  aDlog4irD 
-  n  x2 

W(t,x)  “  X  —  mDlogt  +  nDlog4frD 
Row,  ‘  ' 

0*(t,x)  -  i^i(t,x)  -  *  e)  *  ^W(c(T>-t))  +  a01og4s0 


(4.2.24) 

(4.2.25) 


(4.2.26) 


(4.2.27) 

(4.2.28) 
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V*(t.x)  -  "  T^)  ^  “  log(T-t)) 


■T 


W*(t,x) 


-  — f - 

2  lt<T-t)J 

mxf  T-2t  1 
"  2  Itd-Oj 

Thus,  by  Prop.  4.2.fl,  , 

*  t]  ^l°e(t(T-t:))  +  BDlog4«Dj 


AH  (t.x)  - 


.K*,.  mfl-2ti. 

AV  (t.x)  -  2t(T-e) 


-  exp 


‘f¥l 

r  T-2t  ] 

lt(T-c)| 

1  ®D,  1 

1 

tel]l 

*  J 

satisfies 


(4.2.29) 


<»  -*^.7  ^  r  T  4T  1  :7 

at  "  2in  ^  [4  [t(T-c)J  ■  t(T-t)J 


(4.2.30) 
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CONTINUOUS  PROGRAMMING  AND  NONLINEAR 
FILTERING  OF  QUANTUM  CONTROLLED  PROCESSES. 

V.P.  BELAVKIN 

Moscow  Institute  of  Electronic  Mashinebuilding 
B.  Vusovski  3/12,  Moscow  109028 

A  quantum  continuous  Bellman  equation  is  derived  for  the  solution  of  the  pro¬ 
blem  of  optimal  control  of  a  quantum  stochastic  process  with  nondemolition  measure¬ 
ments.  The  solution  of  this  equation  u^it.u^’.p)  together  with  the  solution  of  the 
corresponding  nonlinear  filtering  problem  p  ■  n(t)  defines  the  optimal  control 
strategy  d°(t,z^,q(t))  =  u°(t,u^,lT(t)) . 

Let  us  consider  a  quantum  controlled  process  over  the  algebra  6  =  B(E)  described 

by  the  family  of  normal  representations  i(t)  :6  -t-*  where  &|.=6®8{F^)  , 

2 

=  A  (jC  ([0,t[)  is  the  Fock  space. 

Let  S  *  tl(t')  be  a  Hausdorf  space  of  controlling  processes 

t'<t 

u*-  “  {u(t')|t'€[0,tt}  such  that  «  U®  =  (1^*®  for  all  t.sER^  ,  where 

U®  c  »  U(t+T)  and  =>  ,  U  »  .  We  consider  a  quantum  controlled  pro- 

^  0jT<S 

cess  i(t,u^)  =  i(t)(u^)  over  the  algebra  S  »  B(E)  with  respect  to  =  S®  B(F^)  , 
F^  is  the  Fock  space  over  £^(t0,t()  described  by  the  Hudson-Parthasarathy  dynamical 
equation  (1)  for  P(t,u*-)  ”  i(t,u^,p) 

dP-T(u,P)  ®  Idt  -  2ReB(P)®dA  +  A(P)  »  dN,  P(0)  -  p  €  6  , 

where  A  -  A(t,u*’),  B  -  BCt.u*"),  Y(u(t))  -  T(t,u*^,u(t))  are  defined  in  standard  way 

by  operator-valued  functions  V(t),  X(t)  :  -*  A(t,u^)  -  i(t,u^,5)  with  unitary 

V(t,u^)  and  self-adjoint  H(t,u*-,u(t)) .  We  shall  suppose  that  the  controls  u^.  G 

are  defined  by  strategies  u^  ■  d^(z^,q^)  •  {d|.(t+T)|T  6  [0,s[}  where  z^“(u^,q^)  , 

q  «  q“  q^  -  q^  ,  q^  »'{q(t+T)|T  €  (0,s(}  are  the  results  of  nondemolition  mea- 
^  o  t  s  [t  t+8[ 

surements  on  the  interval  [tit-i-st,  <1^^  €  R  ’  described  by  a  coomutative  pro- 

cess  Q(t)  satisfying  the  equation  respecting  to  dY  ■  X®Idt+V»dA,  dfl  ■  X  X*Idt  + 

*  2ReX*VedA  +  ladN 

dQ-g(u(t))®  Idt  -  2Re(b*  DdY*  (f*I)dn  ,  Q(0)  -xl 

with  b(t,u^),  f(t,u^),  g(t,u^,u(t))  €  A(t,u*-)’  . 

Let  us  consider  the  optimal  control  problem  with  the  operator-valued  risk 
R»(u)  6  A^(u)  “  V  A(t4’S,u***)  ,  satisfying  the  equation 
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®t  (“)*['  S(t,u®,u(t))dt  ♦  Rj  (tt)  , 

o  J  Cj,  1 

where  S(t*u^,u(t))  CA(t*u^)  for  ell  t€R^  .  The  opclmel  conCrol  stretegy  is  de¬ 
fined  as  a  solution  of  the  extremal  problem 

<p««,  R^Cu^.d^iz^.q^)) )  «  inf  , 

where  p  is  an  initial  state  on  &  and  u  is  the  vacuum  state  on  8(F), F  -  F“  .  This 
solution  can  be  found  by  the  quantum  dynamic  programming  method  as  the  solution  of 
the  following  Bellman  continuous  inverse-time  equation. 

THEOREM.  Let  r(t,z*,dj)6S  be  the  averaged  risk  operator  uniquely  defined  by 
i(t,u^,r(t.zt,dj))  -  Et[Rj(u',dt(z^.qj))] 

where  is  the  conditional  expectation  with  respect  to  8^  -  S®  SCF^)  corresponding 
to  the  vacuum  state  on  B(F(.)  and 

r(c,z'^,dj)  -  x^«i(t,u^,r(t,zt,d^))  -  <  irit.z*^)  ,r(t,z*^,d^)> 

be  the  posterior  risk,  corresponding  to  the  strategy  d^,  where  is  the  condition¬ 
al  expectation  on  8^  with  respect  to  the  commutative  algebra  generated  by 

q^  •  {q(t')|t'<t)  .  Then  inf  <  n(t,z*^)  ,r(t>d,.)  >  •  r  (t  ,u^  ,n(  t  ,z*) )  where  the 

dt 

functional  p  •-*  r(t,u^,o)  satisfies  the  following  Bellman  equation  : 

-3jr(p)  •  inf  {<  o,s(u)  >  +  «  p«y(u),6>  P»a,6>2)r(p)) 

u€(i(t)  ^ 

where  3^  ■  3/3t,  6«  6/6p  ,  x  €  fi  :  i(t,u^,x)  ■  X(t,u^), 

P«Y(u)  «  iCo,h(u)  ]  +  -i  ([xp,x*  1  ♦  [x,px*])  , 

p»a  ■  2Reb(x -<  p,x)  )  p  ♦  f(xpx*  -<p,x*x>p), 

and  8(u),  h(u)  €  4  are  defined  by 

i(t,u^,s(u))  -  S(t,u^,u), 

i(t,u^,h(u))  “  H(t,u^,u) 

»  t  . 

and  n(t,z  )  is  an  posterior  state  on  S  satisfying  the  nonlinear  filtering  equation 
dn-  tt.Y(u)dt  -  n»ad?f/(|b|2  ♦  f2  (i;,x*x>)  ,  nCO)  -  p, 
where  <i(lf -  2Rebd?  +  f<iPf,  dY  -  dY -(  n,x)dt,  dW  -  dn  -  (tl,x>dt  . 

In  particular,  for  the  Brownian  observation  {f"0) 

-a^rCp)  -  inf  {<  p  ,f(u))  ♦  ((p^(u),6)  +  2(Re0(x-(  o.x))p,6>2)r(p)  } 


• 
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where  6  ■  b/|b|  and  for  the  Foistonlan  observation  (b“0) 

-3fr(p)  •  in£{<p,s(u)>  +  (<  p«Y(u)  ,6  >  +  4  (  p-xpx*/<  p  ,x*x>,  6)2)r(o)  }  . 

The  linear  dynamical  programming  for  Gaussian  p  and  canonical  x  was  consi¬ 
dered  in  (2) ,  and  the  general  formulation  of  qtiantum  dynamical  prograaming  for  the 
partially  observable  controlled  quantum  objects  in  operational  approach  was  given 
in  (3)  . 
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Risumi :  A  la  mdmoiie  de  Richard  Bellman,  nous  pidsentons  les  contiOles  bipolaiies  en  biologic.  De  par 
ses  settles  applications  thdrapeutiques  aux  domaines  des  tumeurs  cdrdbrales  et  de  la  cancdrologie,  cette 
mdthodologie,  liant  rautomadque  a  la  regulation  biologitjue,  aurait  certainement  eu  ses  faveurs.  Nous  en 
montrons  toute  la  richesse  en  ouvrant  d'autres  perspectives  qui  justiHent  pleinemcnt  le  lien  entre  les 
mathematiques  et  la  medecine  qui  intdtessait  tant  Rklu^  Bellman. 


Abstract ;  In  memory  of  Richard  Bellman,  we  present  bipolar  controls  in  biology.  From  its  therapeutic 
applications  in  the  field  of  cerebral  tumors  and  cancerology  alone,  Richard  Bellnm  would  have  certainly 
been  in  favour  of  this  methodology  which  links  control  theory  to  biological  regulation.  We  show  all  its 
richness  in  opening  other  prospects  that  entirely  justify  the  link  between  mathein^cs  and  medicine  which 
interested  him  so  much. 


1.  INTRODUCTION 


Depuis  maintenant  plusieurs  ddcennies,  de  nombreux  chercheurs  ont  pensd  k  cr6er  un  lien  entre  les 
math^matiques  et  la  mddecine  ( cf.  les  livres  rteents  de  Winiree  1 26  ]  et  de  Swan  [  2S 1 ),  en  particulier  par 
les  essais  de  moddlisation  de  certains  phdnomines  biologiques  et  par  exemple,  en  cancdrologie  ,  par  la 
recherche  de  procedures  medicamehteuses  (chimiotMrapie)  ou  par  la  mise  en  place  de  protocoles 
d'emission  de  particules  actives  spdciflques  (radiothdrapie).  Ils  souhaitaient  ainsi  reunir  la  thdorie 
mathematique  et  la  pratique  medicate.  L'automatique,  appliquee  k  certaines  regulations  biologiques,  repond 
k  cette  exigence  et  k  cette  esperance. 

En  biologie,  de  nombreuses  regulations  font  appel  k  plusieurs  agents  aux  actions  couptees.  n  en  est  ainsi  de 
la  regulation  de  lliydratation  cellulaire  ou  du  contrftte  de  la  mitose  dans  lesquels  interviennent  tes  corticoldes 
d'une  part  et  la  vasopressine  d'autre  part,  de  meme  que  I'insuline  et  le  glucagon  regulent  I'activite 
glycemique.  La  faillite  dans  certaines  pathologies  des  tbenqreutiques  consistant  k  administrer  une  seule 
hormone  trouve  son  explication  dans  le  fait  que  Ton  a  ndglige  tes  reactions  de  rautie  honnone  qui  intervient 
k  cause  d'un  jeu  subtil  de  feedbacks  orrises.  En  outre,  la  biologie  est  un  doonaine  fotement  non  tineaire  oit 
le  principe  de  superposition  rtes  actions  n'a  pu  coun. 
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Aina,  toute  action  tfa^rapeutique  mesurable  doit  passer  par  une  moddlisatioo  non  lindaire  multivaiiable, 
suffisamment  riche  poor  prendie  en  compte  les  aspects  pidponddranu  des  phdnomhnes  dtudids,  et  assez 
simple  pour  envisager  d’une  manihre  raisonnable  les  possibilitds  de  ccmmande  de  ces  systhmes  et  en 
ddduire  les  actions  didrapeutiques.  A  cause  des  couplages,  les  solutions  proposdes,  par  leur  caractbie 
faussement  paradoxal,  peuvent  surprendre,  ddranger,  voir  piovoquer  des  hostilitds.  Pourtant,  les  idsultats 
cliniques  sont  Ih,  authentifids  par  les  radiographies  et  les  scanners,  et  on  doit  espdier  que  les  deux  exetnpies 
que  nous  allons  traiter,  permettent  de  convaincre  de  la  ndcessitd  de  ddvelopper  rapidement  le  champ 
d'acti(»  des  thtopeutiques  bipolaiies  dont  Bemaid-Weil  est  h  I’origine. 


II  LE  SYSTEME  SURRENO-POSTHYPOPHYSAIRE  ET  LA  VASOPRESSINO- 
CORTICOrrHERAPIE 


Dans  le  cadre  de  I'application  de  I'automatique  aux  tiaitements  chimiothdrapiques  en  cancdrologie, 
Sundareshan  et  Fundakowsld  [  24  ],  s’intenogent  sur  le  caractdie  dual  de  I'objet  de  ces  thdrapeutiques  et 
souhaitent  (rouver  des  agents  qui  soient  capables  de  ddtruiie  les  cellules  malignes  tout  en  prdservant  les 
cellules  saines.  En  fait,  au  sein  de  rorganisme  existe  un  important  systdme  qui  assure  la  rdgulation  du 
ddveloppeiiient  cellulaiie  tant  au  point  de  vue  de  la  mitose  que  de  lliydraution  de  la  cellule,  c'est  le  sysidme 
hormonal  sundno-posthypophysaite. 

Le  sysidme  sundno-posthypophysaire,  forind  par  les  cortico>surrdnales  d'une  pan  et  par  la  neuro- 
posthypophyse  d'autre  part,  intervient  ainsi  au  premier  chef  dans  les  manifestations  cliniques  observdes 
chez  le  neuro-chirurgicaL  Ce  systdme  est  responsable  de  manifestatitms  aussi  diverses  que  certains 
oeddmes  du  cerveau,  certains  coUapsus  cdidbraux  aggra^t  les  suites  d'intervention  pour  hdmatome  sous- 
dural,  et  intervient  dans  I'dvolution  des  tumeurs  cdrdbrales  malignes. 

La  reconnaissance  du  couplage  entre  ces  deux  glandes  date  des  anndes  30  ( cf.[  23  ] ),  et  ce  systdme,  aux 
actions  agt^antagonistes  ( cf.[  4, 6, 7  ] ),  assure  (tea  idguladons  majeures.  Ainsi,  la  cortisone,  secidtde  par 
les  cortico-suiidnales,  est  un  merveilleux  agent,  non  seuiement  contre  rhyperhydratarion  cellulaire  mais 
aussi  comme  produit  anti-miiotique,  coinme  cela  adtd  ddmonttd  in  vUro  aussi  bien  dans  le  cas  de  tumeurs 
cdidbtales  malignes  en  culture  que  dans  cdui  de  toute  autre  lignde  cancdieuse  en  culture  de  tissu.  Quant  d  la 
vasopressine,  seodtde  par  la  neuro-posthypophyse,  elle  est  responsable  de  la  idabsocption  de  I'eau  par  le 
tube  tdnal  et  est  un  facteur  de  croissance  tout  k  fait  impoitant  Ce  premier  facteur  de  croissance 
polypetidiqoe  a  M  ddcouven  en  1968  par  Bernard- Weil,  Dalage,  Olivier  et  Piette  [  9  ]  et  leur  rdsultat  a  dtd 
confiimd  ultdrieutement  par  les  auteurs  amdricains,  Rozengurt  et  all.  (  20  ],  en  1979,  et  Monaco 
et  alL  [  18  ]  en  198Z  Nous  renvoyons  d  Pawlikowsid  [  19  ]  pour  avoir  un  rappel  rdcent  des  actions 
miiogdniques  des  neurc^teptides.  Le  ddsdquilibie  entre  les  cordctAles  et  la  vasopressine,  avec  un  excds  de 
vasopressine  favcrisant  le  ddvelpppement  tumoral,  a  dtd  de  nouveau  mesurd  rdcenunent  en  cancdrologie 
digestive  ( cf.  [  11  ] ),  mais  il  a  dtd  constatd  dans  bien  d’anites  cu.  De  plus,  d  cause  du  couplage  entre  ces 
deux  hotmones,  certains  oeddmes  cdrdbraux  rdsistent  d  la  cortisone  et  les  tumeurs  cancdteuses  ne  sont 
vniineiK  influencdes  par  les  corticoides  que  pour  un  court  laps  de  temps  et  avec  des  dotes  trds  dkvdes  de 


CCS  hormones.  Piie  encore,  roiganisme  malade  $e  place  dans  une  position "  dlionidostasie  patbologique" 
( cf.  Bernard- Weil  [  4  ] )  et  ce  ddsdquilibre  rdguld  bdndficie  de  sauvegardes  biologiques  puissantes  qui 
tendent  &  le  mainlenir  en  I'dtat  comme  s'il  s'agissait  du  foocdonnement  physiologique "  normal''. 

Cest  ainsi  qu'est  prdservd  le  ddsdqoilibre  vasopressine-cordcoldes  chez  le  malade  cancdreux, 
radministradon  de  corticoldes  ayant  pour  effet  d'augmenter  le  taux  de  vasopressine  poimant  ddji 
ancrmalement  dlevd  ( cf.  [  3  ] ). 

La  solution  consiste  done  k  envisager  radministradon  simultande  de  vasopressine  et  de  corticoldes 
(  cf.[  S  ] ),  un  module  multivariable  non  lindaire  venant  conforter  les  intuitions  premidres  du  mddecin 
( cf.[  4.  6, 7  ] ). 

Ce  moddle,  reprdsentd  par  un  systdme  diffdientiel  non  lindaire  I  deux  entrdes,  Cj  et  ej,  et  deux  sorties,  Zj  et 
Zj,  peut  s'dcrire  sous  la  forme  suivante  ( cf.  [  IS  ] ) : 

.  3 

H  "  X  [  ( u+p )‘  +  Ci  ( v4q )'  ]  +  e, 

i«l 

V  “  X  [  ki  (  u+P  )■  +  C;  (  v+q )'  ]  +  Cj 
■•1 

X=ei 

(2.1) 

Y-e, 

Zi  =  H-  V 

Z2  =  mLog[(  H+V  )/m] 

avec  H  ■  x+X  ;  V  ■  y+Y,  ou  x  et  y  ddsignent  lespectivement  les  actions  des  corticoldes  et  de  la 
vasopressine  endogdnes  et  X  et  Y  les  actions  des  hormones  exogdnes  ( thdrapeutique). 

D  s'agit  d'un  ddveloppement  en  sdrie  dans  lequel  apparaissent  des  expressions  antagonistes  u  ^  K-V  et  des 
expressions  agonistes  v  «  m  Log  [(  H-^V  )/m  ]  -t-  6(  t ),  avec  6(  t ) «  A  B  sin(  cot )  -f  C  cos(  cot ),  oil  les 
constantes  A,  B,  C  et  co  ( to  >  2n  /  24  dans  un  lythme  dreadien )  ddtenninent  le  s;^hroniseur  6(  t )  lid  aux 
rythmes  biologiques.  L'introduction  de  la  puissance  cubique  se  justife  par  les  conditions  de  stabilitd  du 
systdme  ( cf.  [  4  ] ) ;  p(  t )  repidsente  un  possible  stimulus  osmotique  ;  q(  t )  correspond  &  un  dventuel 
stimulus  voldmique  (  bdmorragie  par  exemple)  ou  un  stress ;  les  paramdtres,  k^,  c^,  li^,  (i  >1,2,3)  sont 

constants  ;  le  paramdtre  m  est  ptis  en  gdndral  constant  ( m  «  0,8  )  mais  peut  aussi  dtte  considdrd  comme 
variable  dans  le  temps.  Lorsque  q  a  des  valeurs  positives,  par  forte  augmentation  de  la  voldmie  par 
exemple,  et  telles  que  x  et  y  deviendniem  ndptifs,  on  prdvott  la  poasibiUtd  de  faiie  quitter  h  m  la  valeur  0,8 
pendant  le  transitoire  ndcessaiie. 

Le  systdme  est  dcrit  dans  un  systdme  d'unitd  commune  ( u.c. )  pour  lequel ; 

0,4  U.C.  >  77  ng/tnl  de  cortisol  ( F )  >  1,1  |iU/ml  de  vasopressine  ( VP ), 

valeurs  qui  correspondent  &  la  moyenne  des  valeurs  expdrimentales  des  rythmes  circadiens  de  ces 
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hormones. 

Les  valenrs  x,  y,  X,  Y  peuvent  fine  assimiUcs  k  ^  concentrations  hoimonales  et  sont  ainsi  snjeties  k  des 
contraintes  de  positivitf.  Dans  le  cas  physiologique  (X«0,  Y>0;p>0,  q^O).  rdquilibration  est 
simulde  avec  un  chanq)  paiamdaique  de  (  2.1 )  donnant  un  cycle-limite  tel  que  le  couple  (  u,v )  admetie 
I'oiigine  (  0,  0  )  comme  point  critique.  L'dquilibnuion  (X>:0,  Y-0)  devient  patfaologiqim  si  nne 
modification  du  phamp  ( 2.1 )  permet  h  un  nouveau  ^le>limite  d'qqiaraltie. 

Les  paramitres  Iq,  Cj,  kj,  c-  ( i  >  1, 2,  3 )  pour  le  systtaie  simulant  la  pathologie,  et  c,.  kj,  Cj  pour  le 

syst&me  simulant  le  cycle  physiologique,  s<»t  idendfids  h  partir  des  donndes  cliniques  et  physiologiques,  i 
I'aide  de  la  mdthode  d'intdgradon  numdrique  de  Davidon-FIetcher-Powell  avec  contraintes  ( cf.  [  1  ] ).  Le 
critdre  k  minimiser  }  ( Iq,  Cj,  Iq,  c-,.  T )  est  donnd  par : 


3  <  ki,Ci.i£;.c;.T)=x  [ ( (^.2) 

j 

ou  X  et  y  ddsignent  des  valeuts  expdiimentales  et  x  et  y  les  solutions  "endogdnes”  du  systdme  ( 2.1 )  dans 
lequel  on  a  ptis  X  =  O.Y  =  0,  p  =  0  et  q  =  0.  La  quandtd  T  correspond  k  trois  cycles,  soil  ici,  h  72  heures. 
Dans  le  cas  d'une  homdostade  pathologique,  la  "simuladon  thdrapeudque"  consiste  h  ddterminer  les 
hormones  exogdnes  X  et  Y  de  fa^on  i  ramener  le  systdme  dans  une  posidon  dliomdostasie  physiologique. 
Une  premidre  mdthode  ( cf.  [  6, 7  ] )  consiste  i  dcrire  les  entrdes  e^  et  ej  dans  une  forme  semblable  h  celle 

des  hormones  endogdnes,  soit : 


3 

«i  =  Z  (u  +  py  +  Cj^i  (v  +  q)i]+X,(X.o,)  +  X2(X  -Oi)2  +  X3{X-Oi)3 

'  i>I 

'  (2.3) 

i  3 

ej  =  X  f  3+i  (u  +  P)‘+  Cj^i  (v  +  q)']+Xi(Y-o,)  +  X2(Y-eq)2  +  X3(Y-a,)3 

iml 

$ 

\  .  avec  ^j,  Xj,  X3,  Xp  X3,  Oj,  Oj  des  paramdtres  constants  ayant  pour  r61e  d'dviter  la  ddiive  du  cycle- 

limite  de  dimension  4  que  suivent  les  quatre  dtats  du  systdme.  On  idendfie  alcors  les  paramdtres  de  ( 2.3 )  d 
I'aide  de  la  mdthode  de  Davidon-FIetcher-Powell. 

Remarque  1:  La  tentadon  de  prendre  pour  les  entrdes  ej  et  ej  la  dilfdrence  entre  les  dquations  de  I'dtat 
physiologique  et  de  I'dtat  pathologique  conduit  &  un  contrdle  qui  peut  ne  pas  sadsfaire  les  condidons  de 
i  I  posidvitd  des  variables  x,  y,  X,  Y,  ni  assurer  I'existence  (fun  cycle-limite  ( cf.  [  6  ] ). 


Une  seconde  mdthcxie,  basde  en  premier  ( cf.  [  15 1 )  sur  le  (idcouplage  et  la  lindarisadon  des  systdmes  non 
lindaires  ( cf.  [  12, 13, 14  ]  et  les  bibliographies  affdrentes ),  consiste  en  fait  k  inverser  le  systdme  ( 2.1 ) 
{cf.(16,17]). 


A  panir  du  systimc  ( 2.1 ),  on  considiie  alois  les  relations  suivanies : 
H  -  1/2  (  z, 

V  =  1/2  (  -Zj+me^*2^™^  ) 

X  »  K  -  X 

Y  *  Y  -y 

X  et  y  dtant  solutions  des  Equations  diffdrendelles : 

3 

*  =X  [lq(Zl+P)‘+Ci(Z2+  ®  + 

i-1 

3 

y-S  [  ki(Zj  +  p)‘  +  ci(z2  +  0  +  qy] 
i>l 


(2.4) 


(2.5) 


n  s'agit  done  de  pennettre  aux  sorties  Zj  et  Zj  du  systftme  ( 2.1 )  de  passer  de  la  position  pathologique, 
donnde  par  les  Equations  diifdrentielles : 

3 

Vl  =  2  [  (  •  V  (  Vl  +  P  )‘  +  (  C;  -  cj  )  (  '  ■  +  q )‘  ] 

i>l 


avec  V  »  Yj  +  A  +  B  sin(  (Ot )  +  C  cos(  cot )  et  ca  >  2x  /  24, 

it  I'dquilibre  physiologique  ddcrit  par  les  Equations  diffdrentielles  obtenues  k  partir  des  donndes 
expdnmencales : 

3 

[  (lCi-JEj)<p;  +  (c'i-ci)v‘] 
i-1 

^-(2  [  (JCi+^)<^-»-(^  +  ci)  v‘]  )  .  (2.7) 

i-l 


avec  v«<P2-t’A-«-Bsin(aK)-*-C  coU.  ok  )  et  oo  >  2x  /  24. 
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On  toit  Zj  et  ^  SOUS  la  fixme : 

Zj«8i+<Pi 

(2.8) 

Z,-52  +  <P2 

Le  souhait  du  didi^wate  est  alors  de  trouver  des  foncdons  5j  et  £2  qui  pennettent  en  premier  de  ddfinir  un 
transitoire  amenant  les  couibes  pathologiques  inidales,  leprdsentdes  par  x  et  y,  vers  les  courbes 
physiologiques  que  doivent  suivre  les  vsoiables  H  et  V.  somme  des  actions  des  hormones  endogbnes  et 
exogbnes.  En  second,  aprbs  la  pdtiode  tnnsitoire  ( deux  i  trois  jours ),  le  thdrepeute  souhaite  &  la  fois,  voir 
s'installer  un  idgime  pennanent  aussi  procbe  que  possible  du  lythme  drcadien  pbysiologique  pour  les 
vaiiables  Het  V,etmettreen  place,  pour  denombreuses  raisons  fadleshdeviner,  one  action  tfadrapeutique 
pdriodique  de  pdiiode  dgak  id  1 24  heures. 

Cependant,  I'analyse  tmmddiate  des  Equations  ( 2.5 )  montre  qu'avec  les  coefficients  du  pathdogique,  il  n'y 
a  aucune  raison  pour  que  Vintroduction  dans  ces  Equations  des  rythmes  physiologiques  entralne 
rappaiition  d'un  cycle-limite.  Bien  au  contraire,  comme  le  confirment  les  simulations  numdriques,  on 
assisie  h  one  ddiive  affine  du  cycle.  La  demonstration  de  ce  phenombne  itant  dvidente. 

Ainsi,  la  seule  possibilitd,  en  tdgime  permanent,  est  de  ddfotmer  aussi  pen  que  possible  le  rythme 
pbysiologique  pour  assurer  la  pdriodidte  de  la  thdiapeutique  rq>r6seniee  par  X  et  Y,  les  fooctions  6]  et  Sj 

dtant  aloes  elles  aussi  pdriodiques.  On  est  conduit  ainsi  h  rdaliser  une  optimisation  sous  les  contraintes 
x20,y20,X20,Y20.  Enfin,  il  faut  s'assurer  que  le  cycle>limite  obtenu  est  stable  et  que  le  systbme 
est  en  plus  structurellement  stable. 

D  est  i  noter  que  le  prindpe  de  I'utilisation  de  ropthnisation  est  judicieux  au  regard  de  la  notion  de  rythme 
pbysiologique  moyen  qui  est  udlisde  et  aussi  vis  h  vis  des  inceititudes  qu'ambne  I'utilisation  d'un  module. 
Ainsi  les  functions  Sj  et  Sj  doivent  pennettre  de  satisfiure  les  conditioos  de  positiviid  des  variables  x,  y, 

X.  Y,  et,  aprbs  un  transitoire,  doivent  assurer  I'existence  d'un  rdgime  permanent  cyclique  et  basd  sur  le 
rythme  circadien.  D  s'agit  alors  de  trouver  une  classe  de  fonctions  suffisamment  riche  pour  pouvoir  cootenir 
les  solutions  cherchdes.  On  peut  envisager  une  recherche  hybtide  en  sdparant  la  partie  transitoire  du  rdgjme 
permanentOn  peut  alon  considdrer  la  classe  de  fonctions  h  quaire  paramdtres,  dense  dans  I'ensemble  des 
fonctions  continnes  sur  tout  intervalle  compact,  et  ddfinie  par : 


f*  +  •  _ b 

“Jo  TTdi . 


—  cos(e*)dt  ♦€  avec  d>0 
'  cos  (bt ) 


Gene  classe  de  fonctions  est  utilisde  par  Boshetnitzan  [  2  ]  dans  la  recherche  des  dqnations  difforentielles 
universdles  ( cf.  [  2, 22  ] ). 

Cette  secoode  mdthode  est  en  court  d'dtude. 
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Remarque  2  :  On  pounait  s'inqukiter  de  I'inqxxsil^td  de  nouver  un  cootiMe  tbdnqteutique  cqwble  de 
idtablir  les  tydunes  ^yskdogk|aes,  mais  on  nn  doit  pat  oublier  qne  dans  la  idalkd  les  panuntoes  qni 
dfliBBWttoiit  ie  ertwi^^  dn  syittafe  sont  vaiiables  Ot  si  fis  soot  pasads  da  la  posituin  jdorsialai^qnB  i  la 

shuuion  patfaolc^ne,  la  didn^ta,  dans  tescaas^  afvenfldM,  poaoda  ^*01  maiatien  fixed  d!tm  tytiinia 
jxochB  du  lydunai^iyincdi^que,  pendant  nna  pdikida  suffisimta,  pemwtBa  anx  panmtoas  de  se  lecakr  sur 
nxmidostasia  {diyitioloifipie- 


nLLE  CX3UPLAGE  INSULINE-GLUCAGON  ET  LE  DIABETE. 

L'activitd  glycdmiqne  pent  fitre  considdide  comma  la  idsnltante  des  actions  amagonistes  du  glucagon 
hyperglycdtmant  at  de  I'insMine  hypoglycdmianie,  ces  deux  honnones  apssant  (fune  fapon  couplde.  Ce 
systdme  prdsente,  par  rapport  an  systdme  sundno-posdtypophysaiie,  nne  pankulaiitd  remarqnable  au  plan 
anatomique.  Dans  le  cas  de  la  idponse  glycdmique,  la  natnre  a  instaUd  le  mdcamsme  de  commande  de  la 
regulation  dans  un  rndme  endidt  •  les  Qots  de  Langeriums  -  au  sein  du  pancidas.  On  irouve  dans  ces 
cellulaires  la  fabrication  simultande  de  rinsuUne  et  du  glucagon  sons  Taction  coordonnde  de  la 
somaiDstatine.  Devant  les  idsultats  cliniques  obtenus  d  Taide  de  la  vasopiessino<Qnicoifadra]w.  il  semblait, 
an  regard  des  enjeux  en  diabdtologie,  interessant  de  proposer  une  moddlisadon  du  systdme  insuline- 
glucagon  sous  Tangle  de  la  vision  bipolaiie  des  systdmes  ago-antagomstes  ddfinis  par  Beinaxd>WeiL 
La  moddlisadon  proposde  prend  la  forme  d'on  systdme  diffdrendel  non  lindaire.  I  trois  entries 
ei,e2. 63  ettrois  sorties  Z},Z2. 23  ainsi  ddfini  ( cf.  {  8  ] ) : 

3 

K-X  m(W’V+ejy+Ci(H  +  V-my]+e, 

i-l 

3 

V"X  [  ki(H  -  V +e3)‘  +  &j(H  +  V -my] +e2 

i«l 

X-ej 

Y-Cj  (3.1) 

6-fi,(Go-G)  +  g2[g3(th(g4(H  -  V+Y-X  +  gj))  +  th(g4(X-Y  +  gj))-2th(g,g5)]+e,] 

Zi-H- V 

Z2»  K  V  -m 

I3  >0 
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avcc  H  f>x  +  X,V  «y+Y.oik  x et y  d^signent reipecttvaoeiit les  acdoiu du glucagon et de  I'insuline 
endo^aigiatXetYlesactioiudeshoemopeacxogtae8(thdnj)euilque). 

Gg  «  0,78,  ddsigne  le  taux  de  base  physiologique  de  laidpcnse  glycdmique  G(  t )  et  m  <■  2,1. 

Le  systbme  est  doit  dans  un  systbmed'anitd  commune  pour  lequel  one  unitd  conuxuine  vdrifie : 

10  (lU/ml  d'insuline  >  100  pgAnl  de  glucagon. 

Dans  le  cas  de  I'dtude  du  test  de  toldiance  au  glucose,  I'entrde  63  >  p(  t  ),  qui  est  lide  &  la  piise  orale  de 
100gde^ucose,estiiqnd3emdeperlafonctioo : 

P(t)»(p,/(p,-p2)).  IOO.P3  [exp(-p2t)-exp(.p,t)]  (3.2) 

Cooune  pour  le  moddle  du  systdme  sundno-pothypophysaire,  les  paranodnes  des  dquations  ( 3.1 )  et  ( 3.2 ) 
ont  dtd  idendfids,  k  I'aide  de  la  mdthode  cfoptinusadon  non  Hndaiie  de  Davidon-Fletcber-Powell,  k  paidr  des 
couibes  expdrimentales.  Les  paramktres  ddfinissant  la  fonction  p(  t )  ont  dtd  ajustds  une  seule  fois  car  les 
condiiioas  d'absorpdon  intestinale  du  glucose  sont  moins  influencdes  par  les  anomalies  hormonales  que  les 
auties  processus  du  mdtabolisme  gluddique.  Par  contre,  bien  entendu,  les  panmdtres  de  I'dquation  ( 3.1 ) 
sont  k  identifier  dans  le  cas  physiologique  et  dans  le  cas  pathologique. 

La  lecherche  du  contiOle  ( thdiapeutique )  visant  k  coniger  les  anomalies  de  la  rdponse  glycdmique  chez  le 
diabdtique  a  dtd  obtenue  dans  un  premier  temps  ( cf.  [  8  ] )  en  prenant  les  entides  ej  et  ej  sous  la  forme : 

fkj^iCK  -  V  +  p)‘  +  C3^i(H  ♦  V -my] 
i-t 

3  (  3.3  ) 

-  V+p)‘  +  c^(H  ♦  V -my] 

■-1 


Elies  permettent  de  mettie  en  place  un  contrOle  asymptotique  tendant  k  ramener  la  position  limite 
padiologique  k  la  valeur  physiologique  moyenne  de  la  glycdmie  ( 1  g/1 ),  le  ddsdquilibre  initial  glucagon- 
insuline  avant  la  chaxge  en  glucose,  comme  1‘dquilibre  physiologique,  dumt  des  points  critiques  stables  du 
moddle  physiologique. 

On  peutanssiopdrer  comme  pour  lesystdmesundno-postbypophysaireetconsiddrer  les  relations : 

B*  l/2(Zi-«>Z2  +  m) 


V  ■  l/H-z^  +  Zj  +  m) 

X*B  -  X 


(3.4) 


Y-V-y 

X  et  y  diant  sointioas  des  dquations  difldmtielles : 
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3 

i>I 

(3.5) 

3 

y  *X  I k'i(2i+py+<^(z2y] 

i>l 


0  s’agh  id  de  pennetae  aiix  sorties  ,  Z2  et  Z3  du  systdae  (  3. 1 )  de  passer  de  la  positioD  patfaologique ; 

z,(0):z2(0);G(0)  (3.6) 

&  rdquilibie  physiologique  ”  asymptotique'' : 

Zi-0;Zj»0;G=l  (3.7) 

L'dquilibre  physiologique  devant  bien  entendu  £tre  atteint  avant  I'ingestion  suivante,  soit  dans  un  d6Iai 
d'environ  5  heuies. 

Pour  ddtemnner  la  "thdiapeutique"  -  X,  Y  •  i  iq)pliquer  au  systime  "pathdogique'’  ( 3.1 )  on  peut  alors,  par 
exemple,  udliser  de  nouveau  la  classe  de  foncti(»s  it  quatre  parambtres  donn£e  par  la  relation  (  2.9 )  et 
effectuer  une  optmnsatioii,  sous  les  contrainies  x  2  0.  y  2  0.  X  2  0,  Y  2  0,  en  minimisant  I'dcart  entie  les 
trois  sorties  Zj,  Zj  et  Z3  du  systime  "pathologique”  contr616  (  3.1  )  et  les  trois  sorties  q>{,  *^3 

systbme  "physiologique"  ( 3.1 )  soumis  aux  entries  ej «  0, 62  «  0  et  e3  *  p(  t ). 

Ced  fera  I'objet  d'une  prochaine  dtude,  mais  les  simulations  effecniies  avec  les  entries  ei  et  62  sous  la 

fonne  ( 3.3 )  ( cf.  [  8  ] )  montrent  ddji  qu'une  meilleuie  qjprocbe  de  la  cxairbe  glycdmique  est  obienue  avec 
I'intervention  simultante  des  deux  actions  X  et  Y  (insuline  et  glucagon )  plutdt  qu'avec  I'insuline  seule. 


rv.  CONCLUSION 


Nous  avons  ptdsentd  et  illustid  par  deux  exetnples  une  nouvelle  mdthode  de  recherche  liant  dtreritement 
I'autotnatique  et  la  biologie.  Cette  voie  dmt  Bernard- Weil  est  I'initiateur,  ouvie  un  chaiiq>  d'invesdgadmi 
immense  enpettnettant,p'  unproeddddemodflisation  original  qds’appasente  aux  "dynamicai  metaphors" 
de  Rosen  [  21  ] ,  de  prenore  en  compte  I'aspect  ago-antagoiusiB  qui  intervient  darn  on  grand  nombre  de 
idguladons  biologiques.  Cette  moddlisation,  i  mdme  de  simuler  anssi  bien  le  pathologique  que  le 
physkdogiqiie,  propose  des  oontr&les  bipolaiies  aux  incidences  tfadiyendques  paifois  suiprenanies.  D  n'est 
pas  qnesdoo  qne  rawtomatiden  rentie  dans  les  prddsiaas  mddicales  dont  il  n'a  pas  la  compdienoe,  mais  il 
peat  qnandmtaie  indiqoer,  comme  le  montre  ddji  un  cettaiB  nomface  de  pubUcarions  mddicales  ( cf.  ( 5, 
10, 11  )).<ine  la  piaiiqne  des  thdiapeutiqBes  bipolaiies  diendpwb  pas  SOB  champ  tfippBcation.D  n'est 
pu  doomme  qne  dans  un  avenir  qua  I'an  doit  rendre  austi  proche  qua  possible,  cas  drmapeuiiques 
conduimiitbstigprimefrdtatdesoBfBrancetfnagiaadaciBbied’laeshumaiBs. 
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ABSTRACT :  The  aim  of  this  contribution  Is  to  Illustrate  the 
impact  of  computer  simulation  In  the  field  of  biology  and  medi¬ 
cine.  This  paper  shows  how  systems  analysis,  control  theory  and 
computer  science  can  stimulate  new  approaches  to  interpret  can¬ 
cer,  to  predict  tumor  growth  and  to  optimize  tumor  treatment. 

Starting  with  a  review  of  the  current  biological  knowledge  about 
the  origin  of  cancer  a  computer  model  is  constructed 

-  to  simulate  the  time  behaviour  of  disturbed  cell  growth  control 
circuits 

-  to  predict  spatial  tumor  growth  (2-D,  3-D)  and 

-  to  simulate  different  kinds  of  cancer  treatment  (surgery,  ra¬ 
diation-  and  chemotherapy). 

In  the  long  run  the  aim  of  our  work  is  to  optimize  treatment 
strategies  and  schedules  in  vitro  and  In  vivo  by  computer  simula¬ 
tion  prior  to  clinical  therapy. 


1.  BIOLOGICAL  BACKGROUND  OF  THE  CANCER  PROBLEM 

Cancer  is  a  multistep  process  with  the  stages  of  Initiation, 
promotion  and  progression.  Characteristic  features  of  malignant 
tumors  (1)  are  uncontrolled  proliferation,  invasion  in  adjacent 
normal  tissue,  metastases  induced  to  other  tissues  via  lymphatic 
channels  and  the  ability  to  evade  immune  surveillance.  Though 
cancer  treatment  is  concentrated  on  a  prevention  of  metastases 
(2)  the  central  question  in  the  background  of  research  is:  Which 
is  the  initiating  event  that  is  responsible  for  a  stepwise  trans¬ 
formation  of  a  normal  cell  into  a  tumor  cell?  Recent  investiga¬ 
tions  in  the  field  of  molecular  biology  have  focussed  on  dominant 
cellular  genes  called  “proto-oncogenes*  which  cem  be  activated  by 
tumor  viruses,  gene  amplification,  gene  translocation  and  genetic 
nutation.  In  spite  of  this  progress  (3)  the  main  question  how 
genes  and  the  growth  of  normal  and  malignant  cells  are  regulated 
still  remains  open. 
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Most  of  the  normal  tissues  in  U*e  >.ouy  contain  some  cells  that 
can  renew  themselves  (neurons,  liver  cells,  kidney  cells)  if  a 
tissue  is  injured.  The  division  of  a  cell  into  two  new  ones 
involves  four  stages;  G1  — •  S  — ♦  G2  — ►  M  (G1  is  a  gap  after 
stimulation;  S  is  the  phase  of  DNA  replication;  G2  is  a  second 
gap  period  and  M  is  the  stage  of  mitosis) .  When  the  replacement 
has  been  completed  the  repair  process  stops.  Furthermore,  at 
particular  stages  of  the  cell  cycle  the  cells  may  be  bloclced  by 
drugs  or  agents,  or  they  may  move  out  of  the  cell  cycle  into  a 
resting  phase  known  as  GO  (4). 

In  contrast  to  the  normal  cell  a  tumor  cell  is  theoretically  able 
to  divide  indefinitely.  In  addition  a  different  morphology,  lar¬ 
ger  nucleus,  abnormal  number  of  chromosomes  and  the  formation  of 
new  capillaries  (tumor  angiogenesis)  which  is  associated  with  a 
more  rapidly  growing  tumor  (5)  can  be  noticed. 

For  studying  the  process  of  carcinogenesis  tumors  are  induced  to 
animals  or  to  cell  cultures  (in  vitro).  Cell  cultures  are  not 
only  used  to  study  the  division  of  tumor  cells,  but  also  to 
determine  the  effecc  of  chemotherapeutic  drugs.  During  the  last 
years  a  large  progress  has  been  mace  in  experiments  gaining  hard 
data  about  normal  and  abnormal  cell-growth  control  processes  for 
Instance  of  cell- cycle  phase  durations. 


2.  MODELING  APPROACHES 

Starting  from  basic  biological  test  results  a  large  body  of 
mathematically  oriented  work  applying  mathematics  to  the  field  of 
biology  and  medicine  has  been  published  (6-40).  Unfortunately 
these  models  which  consist  of  complicated  formulae,  are  in  most 
cases  not  completely  understood  by  clinicians.  In  this  dilemma 
the  combined  application  of  methods  of  systems  analysis,  control 
theory,  automata  theory,  computer  sciences  and  heuristics  is  a 
good  link  between  the  diverging  areas  of  medicine  and  mathema¬ 
tics. 
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Our  own  approach  developing  closed-loop  control  circuits  for 
tumor  growth  started  in  1968  (11).  At  that  time  the  subject  of 
consideration  was  focussed  on'  stability  conditions  and  on  the 
interpretation  of  cancer  as  an  unstable  closed-loop  control  cir¬ 
cuit.  step  by  step  the  dynamic  behaviour  of  cell  renewal  control 
loops  (Fig.  1)  was  investigated.  Blockoriented  simulation  lan¬ 
guages  have  been  used  for  simulating  the  macromodels.  As  a  result 
the  number  of  cells  as  a  function  of  time  has  been  plotted  (12). 

Then  oncologists  advised  us  to  consider  not  only  the  time  but 
also  the  spatial  behaviour  of  tumor  growth.  In  a  first  approach 
we  developed  models  at  a  cellular  level  which  described  the  2-D 
behaviour  of  a  normal  cell  inoculated  into  a  nutrient  medium  (in 
a  Petri  dish) .  Next  we  extended  this  approach  and  tried  tc  simu¬ 
late  tumor  growth  in  the  tissue  q£  a  tobacco  leaf  (13). 


D1  D2  D3  R 


R:  Required  tissue  oxygen  (desired  number  of  erythrocytes) 
C:  Number  of  red  blood  cells  (erythrocytes) 

E2:  Production  of  the  erythropoietin  hormone 
D1,  02,  03;  Disturbance 


Fig.  1:  Nultl-loop  control  circuit  of  erythropoiesis 
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After  getting  the  results  shown  in  Figure  2  we  improved  these 
models  by  introducing  distinguished  cell  cycle  phases  (Cl,  S,  G2, 
M,  GO,  N).  Thus,  we  were  able  to  simulate  the  3-D  growth  of  a 
single  dividing  tumor  cell  (14)  inoculated  into  the  center  of  the 
cell  space  of  a  nutrient  medium  at  the  beginning  of  the  simula¬ 
tion  run  (Fig.  3). 

The  introduction  of  distinguished  cell -cycle  phases  was  necessary 
because  chemotherapeutic  agents  and  rays  effect  only  a  very 
particular  phase  of  the  cell  cycle  that  means  they  act  phase 
specifically. 

After  simulating  in  vitro  tumor  growth  the  attempt  was  made  to 
substitute  the  nutrient  medium  by  static  blood  vessels  (15). 
However,  very  soon  it  was  clear  that  a  more  realistic  structure 
of  capillaries  was  desirable  for  simulating  in  vivo  tumor  growth. 


3.  DESIGN  STRATEGIES  OF  A  HEURISTIC  MODEL 

The  modeling  of  complex  cell  growth  requires  a  considerable 
simplification.  Some  of  the  oversimplifying  assumptions  are 

-  constant  volume  of  a  cubic  cell 

-  constant  phase  duration  and  constant  cell  loss 

-  only  horizontal  and  vertical  communication  between  neighboring 
cells 

-  a  limited  tissue  volume  by  computer  facilities 

-  side  effects,  immunologic  reactions,  heterogenity,  drug  resis¬ 
tance  and  the  formation  of  metastases  are  neglected. 

If  you  want  to  construct  a  model  of  high  order,  it  is  necessary 
to  design  a  modular  concept.  In  this  case  it  means  to  design 
modular  structured  subsystems. 
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(i)  You  need  cor.t~ol  models  (Fig.  4)  which  describe  the  cell 
division  of  normal  and  tumor  cells  at  a  cellular  level 
including  experimentally  gained  data  e.g.  of  cell-  cycle 
phase  durations. 

(ii)  Heuristic  cell -production  and  interaction  rules  are  re¬ 
quired  describing  the  cell-to-cell  communication.  For  in¬ 
stance  one  rule  of  the  catalogue  may  say; 

All  tumor  cells  residing  at  a  distance  larger  than  100  iim 
from  the  capillaries  after  the  next  division  step  will 
enter  the  resting  phase  GO. 

(iii)  Cell  movement  is  described  by  transport  equations  (diffu¬ 
sion-,  Poisson-equation) ,  that  means  we  have  to  introduce 
into  the  model  gradients  for  pressure  and  metabolic  com¬ 
pounds  . 

(iv)  To  represent  2-D  and  3-D  simulation  results  computer- 
graphics  software  packages  are  necessary. 


E;  DIFFERENTIATED  END  CELLS 


CELL  DEATH 


fig-  4;  Simplified  cytokinetic  model  describing  the  division  of  a 
normal  cell 
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The  large  body  of  statements,  rules  and  equations  has  been  trans¬ 
formed  into  algorithms .  In  addition  algorithms  considering  tumor 
treatment  (surgery,  radiation-  and  chemotherapy)  have  been  deve¬ 
loped  in  subprograms  written  in  FORTRAN  IV.  To  start  the  simula¬ 
tion  program  packages  the  following  input  data  have  to  be  fed 
into  the  computer  (VAX  730): 

-  notations  about  the  character  of  a  cell  (normal,  malignant) 

-  cell -cycle  phase  durations 
cell-loss  rates 

-  initial  configuration  of  normal  tissue  and  of  tumor  cc)ls 

•  distinguished  data  about  the  kind  of  the  planned  tumor  treai: 
ment . 


4,  SELECTED  SIMULATION  RESULTS 

Numerous  simulation  runs  have  been  performed  by  Dlichting  and 
Vogelsaenger  (15-17)  simulating  tumor  growth  and  different  kinds 
of  treatment.  Some  special  results  will  be  demonstrated  now. 

4.1  Growth  of  capillaries 

The  simulation  of  in-vivo  tumor  growth  requires  a  realistic 
structure  of  capillaries.  Therefore  Vogelsaenger  (16)  investi¬ 
gated  the  question:  Is  the  formation  of  capillaries  a  stochastic 
or  a  regulated  process?  In  (16)  the  assumption  is  made  that  each 
cell  of  an  organ  in  evolution  has  a  special  request  for  oxygen 
and  glucose.  Therefore,  parallel  to  the  formation  of  tissue 
capillaries  are  built  with  a  specific  structure  corresponding  to 
the  required  oxygen  and  glucose.  That  means  from  the  viewpoint  of 
control  theory  the  request  for  oxygen  supply  is  regulated  to  a 
constant  level  by  building  a  special  structure  of  capillaries .  A 
comparison  between  Figure  5  and  Figure  6  shows  that  for  the 
cortex  of  a-  rat  the  simulation  result  is  highly  similar  to  the 
experimental  result  received  by  Bilr  (18). 


Now  the  assumption  is  made  that  a  single  tumor  cell  is  arbitrari¬ 
ly  placed  in  the  tissue  of  the  cortex  at  T=1  unit  of  time.  If 
this  tumor  cell  resides  close  to  a  capillary  it  will  divide  and 
move  in  accordance  with  the  cell  production  rules  (Fig.  7) . 
Further  tumor  growth  is  possible  only  because  tumor  cells  produce 
a  substance  which  is  called  tumor-angiogenesis  factor  (TAF) .  This 
factor  stimulates  nearby  blood  vessels  to  send  out  new  capilla¬ 
ries  (Fig.  8)  which  grow  towards  the  tumor,  penetrate  it  and  lead 
to  further  rapid  tumor  growth.  Recently  great  efforts  have  been 
made  to  attack  cancer  by  trying  to  find  a  protein  which  inhibits 
the  production  of  the  tumor-angiogenesis  factor. 


VENTRICULUS 


Fig.  6:  Vascularization  of  the  cortex  (18) 


4.3  Chemotherapeutic  treatment  in  vitro 

As  pointed  out  in  section  1,  the  cytotoxic  effect  of  chemothera¬ 
peutic  drugs  Is  tested  in  cell  cultures.  These  are  very  good  in- 
vitro  systems  which  ceui  be  simulated  by  a  computer  model.  Figure 
9(a)  shows  a  tumor  spheroid  at  T«2(X)  units  of  time  which  has 
grown  up  from  a  single  tumor  cell  inoculated  into  the  center  of 
the  cell  space  at  T“l  unit  of  time. 


At  T»201  units  of  tine  It  Is  assumed  that  all  proliferating  tumor 
cells  (i.e.  the  outside  rim)  have  been  Uilled  by  a  cytotoxic  drug 
(Fig.  9(b)).  Now  the  remaining  resting  tumor  cells  (GO-phase)  in 
the  neight)orhood  of  the  nutr,ient  medium  are  being  recruited  into 
the  cell  cycle  again,  and  after  a  short  time  of  remission  the 
tumor  spheroid  continues  to  grow  (Fig.  9(c) -(d)).  Therefore,  a 
second  therapeutic  attack  or  a  combined  approach  is  recommended. 
The  task  which  has  been  solved  in  (15)  is  to  determine  the  opti¬ 
mum  time  at  which  the  drug  has  to  be  applied  for  a  second  (and 
more)  timo(s). 


5.  FUTURE  PROSPECTS 

From  the  voluminous  catalogue  of  unsolved  problems  in  the  area  of 
cancer  research  I  think  there  are  three  promising  avenues  of 
future  work  in  the  modeling  field: 

-  Optimization  of  distinguished  methods  and  schedules  of  cancer 
treatment . 

-  Generation  of  a  more  realistic  initial  configuration  of  a  tumor 
by  combining  CT-pictures  (Computer  Tomography)  with  predictive 
models  describing  tumor  growth  and  last  not  least 

-  Consideration  of  facts  which  had  to  be  neglected  so  far  (forma¬ 
tion  of  metastases,  immunologic  reactions,  drug  resistance, 
heterogenity,  side  effects) . 
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ABSTRACT 

The  structure-activity  models  are  primarily  oriented  towards  the 
evaluation  of  molecular  similarity.  The  approach  and  the  model  of 
structure-activity  relations  presented  in  this  report  is  based  on  the 
similarity  parameters  developed  by  use  of  probability  functions.  The 
molecular  structures  are  encoded  as  sequences  of  numbers  representing 
counts  of  paths  of  different  lengths.  The  similarity  index  between 
two  compounds  is  calculated  as  the  difference  between  the  gains  of 
information  derived  through  the  comparison  of  the  corresponding 
molecular  path  sequences.  The  similarity  index  is  used  as  a  basic 
information  for  modelling  the  property  prediction  model.  The  corre¬ 
spondences  between  the  ranks  representing  orderings  according  to  the 
similarity  index  value  are  then  searched  and  expressed  as  correlation 
indices.  The  correlation  matrix  represents  the  source  of  data  for 
clusterisation  of  the  compounds.  Optimal  classification  is  obtained 
after  several  testings  with  different  threshold  values.  The  classifi¬ 
cation  of  a  compound  with  unknown  biological  activity  into  one  of  the 
obtained  clusters  of  compounds  with  known  biological  responses  repre¬ 
sents  the  source  of  data  for  prediction  procedure.  The  method  is 
illustrated  on  a  group  of  bensamldlnes. 
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1 .  Introduction 

Th«  biological  raaponae  prediction  aradels  are  often  based  on  cluster 
analysis  methods.  Cluster  analysis  Is  a  multivariate  tectanldue  that 
Identifies  groups  or  clusters  of  related  objects  In  a  multidimensional 
space  [1.2].  The  classification  la  aimed  towards  the  search  of 
pattern  points  which  correspond  to  natural  and  useful  groups  of 
chemical  compounds.  The  location  of  pattern  point  within  a  cluster  Is 
used  for  seml-quantltative  deterrmlnatlon  of  biological  activity  [3]  or 
'other  physicochemical  property.  This  Is  an  usual  procedure  for 
property  prediction  of  non-avallable  or  not  yet  synthetlaed  compounds. 

The  last  decade  brought  In  the  chemical  literature  many  different 
classifying  algorithms.  As  a  rule  the  results  after  the  application 
of  different  algorithms  to  the  same  data  set  differ.  Consequently, 
the  choice  of  the  classification  algorithm  must  be  done  very  carefully 
according  to  the  nature  of  the  studied  problem.  In  general,  all 
methods  for  Identifying  clusters  In  a  multidimensional  space  contain 
some  heuristic  and  arbitrary  elements.  Quantitative  evaluation  of  the 
accuracy  of  the  classification  method  and  the  prediction  power  Is 
possible  only  In  the  case  where  the  chemical  data  are  abundant. 

Basic  assvunptlon  used  In  development  of  structure-activity  models  Is 
the  expectation  that  molecules  with  similar  structural  features  will 
exhibit  similar  physicochemical  properties  and  biological  or  pharmaco¬ 
logical  activities  [4] .  Structural  similarity  or  dissimilarity  of 
drugs  finds  application  In  quantitative  structure-activity  relations¬ 
hip  (OSAR)  studies  and  In  drug  design  [5].  The  definition  of  the 
similarity  within  the  models  Is  based  on  mathematical  terms,  which 
describe  the  chemical  structure  of  the  drug.  The  most  difficult 
problem  In  modelling  Is  the  derivation  of  mathematical  expressions  for 
chemical  structure  encoding.  The  mathematical  terms  used  In  the  model 
are  expected  to  contain  a  lot  of  Information  and  to  have  general 
applicability  to  different  chemical  systems  under  a  variety  of 
conditions . 

The  model  presented  In  this  report  makes  use  of  the  mathmsaticsl 
property  of  the  molecular  graphs.  The  elasslfleatlon  approach  is 
based  on  the  comparison  of  all  possible  molecule  rankings  (n^resented 
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as  strlncs)  generated  according  to  the  alxkllarlty  of  a  particular 
■dlecule  frcei  the  initial  set.  The  results  of  string  ccetparison  are 
expressed  as  correlation  coefficients  and  used  for  group  generation. 
The  properties  associated  with  a  group  are  used  as  source  data  for 
prediction  of  an  unknown  biological  response. 


2.  The  developed  nodel 

2.1.  Definition  of  the  molecular  similarity  measure 

An  important  problem  in  modelling  structure-activity  relations  (SAS) 
is  the  definlton  of  the  molecular  similarity.  The  similarity  itself 
is  a  mathematical  relation  with  transitive,  reflectional  and  equiva¬ 
lence  properties,  but  the  molecular  similarity  derived  from  the 
mathematical  properties  of  the  molecules  does  not  always  reflects  in 
the  same  manner  these  mathematical  relations  in  the  real  chemical 
world.  In  the  real  word  there  are  other  elements  besides  the  chemical 
structure  that  govern  the  molecular  behaviour. 

The  selection  of  appropriate  molecular  descriptors  in  SAR  is  of  a 
great  importance.  In  the  chemical  literature  it  is  commonly  accepted 
that  major  factors  that  govern  the  chemical  events  and  biological 
activity  are  the  molecular  shape  and  the  molecular  structure  [6].  In 
our  nodel  the  molecular  descriptors  are  derived  from  the  molecular 
graph,  not  from  the  molecular  physico-chemical  properties  [7].  We 
decided  to  use  as  mathematical  descriptors  a  set  of  structural 
parameters  already  found  useful  in  the  study  of  structure-activity 
relationships.  The  molecular  model  used  is  the  structural  formula  in 
which  the  hydrogen  atoms  are  supressed  according  to  the  widely 
accepted  practice  [8] .  The  hydrogen  atoms  are  less  essential  for  the 
chemical  behaviour  and  their  presence  can  be  always  deduced  if 
required.  The  characterisation  of  the  ohoad.cal  structure  is  done  by 
•Bumeration  of  the  self-avoiding  walks  or  paths  with  different  length 
la  the  hydrogen  depressed  gra^.  The  use  of  path  codes  in  the 
dlseosslm  of  similarity  versus  property  or  activity  has  shown  that 
shorter  paths,  in  partioular  the  paths  of  lengths  two  and  three, 
refleet  the  physioeehemieal  properties  of  the  compound  [9]  while  paths 
of  longer  length,  whieh  eneode  the  presenoe  of  structural  details  at 
larger  separations  reflect  the  molecular  shape  and  are  of  interest  for 
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study  ot  th«  blolocioal  activity  [^0] .  Derivation  and  oosvutatlon  of 
salf-avoldlnx  paths  a«y  be  found  alsawhara  [11,12]. 

The  approach  and  the  model  of  structure-activity  relations  presented 
in  this  report  is  based  on  the  similarity  parametres  developed  by  use 
of  probability  functions.  The  sequence  of  path  numbers  for  a  compound 
Ai  may  be  viewed  as  a  distribution  of  a  particular  property  of  the 
molecular  graph  [13].  The  domination  of  the  aise  effect  which  may 
obscure  the  analysis  when  molecules  of  different  aise  are  considered, 
may  be  avoided  by  normalization  of  the  path  sequences.  The  path 
sequences  are  normalized  by  dividing  the  entries  in  every  sequence 
represented  as  a  vector  xi  (xi  =  [xij],  J=l,2,..,m;  m  is  the  number  of 
the  longest  panh)  by  the  number  or  atoms  in  the  structure.  We  denote 
the  normalized  vector  as  xt . 

Following  Jeffrey  [14]  the  similarity  between  two  chemical  structures 
Ai  and  Ak  belonging  to  a  studied  data  set  can  be  defined  as  [15]: 

s 

KAl/Ak)  =  )  (!•> 


where  pi  i  is  the  probability  that  a  randomly  selected  element  of  the 
sequence  Ai  will  be  found  in  the  J-th  group  of  rlements,  paj  is 
defined  analogously. 

In  the  case  of  strings  of  path  numbers  characterizing  a  siolecule,  the 
probability  that  a  randomly  selected  path  is  in  the  group  of  paths 
with  length  J  will  be: 

■ 

pij  =  xtj/(  ^x*j  )  (2) 

As  pij  is  calculated  from  the  elements  which  reflect  the  molecular 
features  of  Ai  the  quantity  I(Ai/Ak)  measures  quantitatively  the 
degree  of  similarity  six  between  the  molecules  Ai  and  Aa .  The 
similarity  matrix  8  for  a  particular  set  of  compounds  is  obtained  by 
calculating  the  similarity  indices  sik  for  every  pair  (Ai,Ak)  in  the 
data  set. 


Th«  vlmilarltjr  index  in  the  developed  model  is  used  as  a  basic 
information  for  propertjr  prediction.  The  studied  compounds  are 
ordered  according  to  the  similarity  index  value  between  the  compound 
Ai  and  all  others  componds  in  the  data  set.  The  ranking  is  obtained 
with  ordering  of  the  structures  by  ascending  values  of  the  similarity 
indices  [15].  After  generation  of  all  rankings  in  the  data  set  a  nxn 
matrix  is  obtained.  The  relationship  between  the  compounds  may  be 
expressed  quantitatively  by  use  of  different  string  comparison  met¬ 
hods:  trace,  alignment  and  listing  [16].  The  calculated  quantities 
give  an  Information  about  the  similarity  between  two  compounds 
"derived"  from  the  slmileirity  relationships  of  both  compounds  to  all 
elements  present  in  the  data  set.  In  that  way  all  individual 
structural  characteristics  present  in  the  data  set  are  fully  conside¬ 
red. 


The  correspondence  between  two  particular  sequences  is  calculated  by 
counting  the  number  of  identities  in  traces  generated  for  these 
sequences  [IS].  A  trace  between  two  sequences  consists  of  lines 
connecting  the  elements  from  both  sequences.  An  element  can  have  no 
more  than  one  line  and  the  lines  must  not  cross  each  other.  If  the 
elements  connected  'ty  a  line  are  the  same  then  the  pair  represents  an 
identity,  if  they  are  different,  the  pair  constitutes  a  substitution. 
The  result  of  the  comparison  between  the  elements  of  the  sequences  A 
and  B  la  expressed  as  the  quantity  W(A,B).  W(A,B)  represents  the 
number  of  different  identities  found  in  all  traces  generated  by 
comparison  of  two  ranks  and  diminished  by  one  [15]: 
n 

W{A,B)  (ci  -1)  (3) 

1  >i 

where  n  la  the  number  of  the  sequence  elements;  ci  is  the  number  of 
identities  in  the  trace  i  on  the  right  of  element  i.  Detailed 
description  of  the  method  of  calculation  of  H(A,B)  is  given  in  [15]. 
The  correlation  coefficient  r  for  two  compounds  Ai  and  aj  is  computed 
according  to  the  expression  [13]: 

nj  =  [2W(Ai  ,Aj  )/n(n-l)]  (4) 

where  n  is  the  number  of  the  string  elements. 


The  correlation  coefficient  rij  is  1  if  the  orderings  of  the  sequence 
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elements  compared  are  identical  and  approaches  sero  if  they  are 
completely  different.  The  correlation  coefficients  nj  for  all 
possible  raxikings  in  the  data  base  form  the  correlation  matrix  S.  An 
element  of  R  gives  a  quantitative  estimation  of  the  similarity  between 
the  compounds  Ai  and  Aj .  Two  similar  compounds  i.e.,  Ai  ,  Aj  will 
generate  always  a  similar  ordering  of  the  rest  of  the  data  set 
elements  and  they  will  have  very  high  nj .  If  the  contrary  is  true, 
then  the  value  of  n j  will  be  low. 


2.2.  The  compound  clustering 

The  developed  clustering  algorithm  makes  use  of  the  correlation  matrix 
R.  The  first  step  of  the  procedure  is  the  search  for  the  moat 
correlated  compounds  within  the  matrix  R.  The  searched  compounds 
represent  the  kernels  of  the  future  clusters.  Each  kernel  in  the  very 
beginning  contains  only  two  compounds  with  the  highest  found  n  j . 
Other  kernel  elements  are  added  according  to  the  first  threshold  value 
defir ad  on  the  base  of  the  first  chosen  value  of  nj .  The  kernel 
elements  are  represented  in  Flg.l  with  the  sign  9  (note  that  this  sign 
represents  two  compounds ,  the  compound  i  and  J ) .  The  second  step 
completes  the  clustering  procedure.  Another  threshold  value  is 
defined  for  classification  of  the  rest  of  the  data  set  components.  An 
element  k  is  added  to  a  particular  kernel  if  the  value  of  r  for  this 
element  i.e.,  rkn  is  bigger  or  equal  to  the  second  threshold  value. 
If  two  or  more  kernels  satisfy  this  condition  then  the  element  is 
added  to  the  kernel  with  the  highest  mew  value  of  rkn .  The  second 
threshold  value  is  lower  than  the  threshold  value  for  kernel  generati¬ 
on.  The  prescribed  values  r  in  both  steps  may  be  changed  during  the 
clustering  procedure  according  to  the  nature  of  the  classified  data. 
Sometimes,  the  first  threshold  value  happens  to  be  too  high.  This 
results  in  a  small  number  of  clusters.  In  this  case  the  threshold 
value  is  decreased.  On  the  other  side,  the  criteria  for  this 
threshold  value  has  to  be  high  enough  because  if  it  is  low  the 
similarity  criteria  can  be  lost.  Optimal  classification  of  the 
compounds  is  obtained  after  several  testings  with  different  threshold 
values  of  the  correlation  index. 
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2.3.  Prop«rt7  pirsdlotloa 

Th«  Mabership  Inforaatlon  lor  a  coapound  with  uaknown  blolosleal 
activlt/  In  a  cluatar  with  known  biolotlcal  raaponaas  la  uaad  lor 
property  prediction.  The  developed  procedure  lor  biological  reaponae 
prediction  aaauaea  that  the  valuea  ol  the  biological  reaponaea  ol  all 
ooapounda  in  the  cluster  contribute  to  the  value  ol  biological 
reaponae  ol  the  unknown  coapound.  The  contribution  ol  a  particular 
coapound  la  the  cluster  ia  taken  to  be  proportional  to  the  degree  ol 
aiallarity  ol  that  coapound  to  the  unknown  one.  The  property 
predlcltlon  procedure  is  as  lollows: 

let  compound  x  be  clasailied  into  a  group  having  n  compounds.  The 
average  of  the  correlation  eoelllclents  ol  the  group  r>  (ra  is 
calculated  as  the  sum  of  nj  of  the  cluster  divided  by  n)  and  the 
average  ol  the  biological  responses  lor  the  group  BSa  (BRa  is 
calculated  as  the  sum  ol  BRj ,  J=l,....,n  divided  by  n)  are  used  lor 
prediction  ol  the  unknown  value  ol  BR  by  the  application  ol  the 
following  equation: 

n 

BRpt  aSi  et as  (x)  =  BRa  ^  ( (BRi -BRa  )  ( l-^mt -ra  )  )/n  (6) 

For  a  large  data  base  calculation  ol  the  similarity  index,  the 
correlation  coelliclent  as  well  as  the  property  prediction  requires  a 
computer  aid.  A  computer  program  has  been  developed  in  programming 
language  Pascal  and  implemented  on  Vax  11/760.  The  procedure  and  the 
model  is  represented  in  Fig. 2. 


3.  Appllcatons 

A  group  of  compounds  which  consists  ol  73  benaamidines  derivatives 
with  dopamine  receptor  affinity  has  been  taken  as  a  basic  data  set. 
The  experimental  values  ol  logP  lor  these  compounds  have  been  taken 
from  the  work  ol  Hansch  and  coworkers  [17],  The  obtained  clustering 
consists  ol  11  groups,  12  compounds  are  not  classified  because  of  low 
values  of  their  correlation  coefficients.  The  correlation  matrix  is 
displayed  in  Fig.l  where  eleven  groups  ol  highely  correlated  compounds 
may  be  recognised.  The  values  ol  logP  presented  on  the  left  aide  ol 
the  figure  are  grouped  very  well  too.  The  predicted  values  for 
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s«lected  eoapounds  ara  shown  in  Tabla  I.  Tha  thrasholds  valuas  hava 
baan  obtatnad  aftar  aavaral  attaopta.  Finally,  as  tha  optimal  tha 
karnal  thrashold  valua  has  baan  takan  to  ba  0.94  and  tha  elustar 
thrashold  valua  0.6. 


4.  Discussion  and  conclusion 

The  mathod  prasantad  in  this  report  shows  that  string  comparison 
tahniquaa  may  ba  applied  in  chamioal  classification  of  compounds  with 
similar  biological  activities.  Tha  developed  mathod  and  models  may  ba 
considered  as  an  evidence  how  certain  mathematical  tahniques  may  be 
applied  for  derivation  of  the  relationship  between  biological  system 
response  and  structure  of  chemical  compound  l.e.,  the  potential  drug. 
The  molecular  path  counts  are  found  as  suitable  non-emplrical  parame¬ 
ters  for  description  of  the  molecular  structure.  The  same  approach 
may  be  applied  to  other  applications  with  other  molecular  descriptors 
and  sequence  comparison  having  different  contents.  Obtained  predicti¬ 
ons  of  biological  responses  are  optimistic  and  suggest  further 
development  of  the  method. 
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Fig.l.  Clusterization  of  73  benzamiCine  derivatives 


Fig. 2.  The  biological  activity  prediction  model 


Table  1.  Predicted  values  of  logP  for  benzaaldine  derivatives 


Coup. No.  i 

Measured  BR  1 

Calculated  BR  ! 

Prediction  error  1 

3  i 

2.35  ; 

2.47  1 

5X  I 

2  1 

2.25  1 

2.48  1 

lOX  1 

8  1 

2.66  I 

3.10  1 

16k  i 

73  1 

3.03  i 

3.02  j 

OX  { 

24  ! 

3.77  1 

4.11  1 

9X  i 

34  { 

4.00  1 

3.92  : 

2X  1 

40 

4.09  ! 

4.22 

3X 

69  ; 

4.66  i 

4.33  ! 

8X  1 
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The  theory  of  feedback  control  as  a  posf  ibie  stabilizing  mechanism  has  already  been  introduced  into 
ecosystem  analysis.  One  problem  in  the  theory  is  the  identification  of  the  informational  links  by  which 
such  controls  operate.  Cyclic  controls,  for  example,  zero-mean  sine  functions  added  to  certain  exchange 
flows  in  the  system,  might  also  contribute  to  system  stability.  Their  advantage  is  that  they  operate  without 
need  for  information  from  the  rest  of  the  system.  The  theory  of  ecosystem  cyclic  control  is  presented  and 
applied  to  data  from  an  oyster  reef  ecosystem. 

I.  INTRODUCTION 

To  address  the  problem  of  ecosystem  stability  and  performance,  the  previous  control  studies 
utilized  solely  classical  control  principles,  feedback  and  feedforward  (Olsen,  1961;  Lowes  and  Blackwell, 
1975;  Mulholland  and  Sims,  1976;  Vincent,  et.al.,  1977;  Goh,  1979;  Hannon,  1985b,c,  1986;  DeAngelis, 
1986).  If  knowledge  of  the  current  output  is  used  to  modify  the  inputs  to  control  the  system,  we  have  a 
feedback  control  situation  (Wonham,  1984).  Feedforward  control  uses  current  knowledge  of  the 
disturbance  (rather  than  output)  as  the  basis  for  a  corrective  action  (Takahashi,  et.  al.,  1970).  The  major 
problem  with  these  lands  of  controls,  however,  lies  in  explaining  how  the  requisite  information  flows 
occur. 

An  alternative  approach  to  ecosystem  stability  is  found  in  the  concept  of  cyclic  (or  vibrational) 
control  (Meerkov,  1980;  Bellman,  et.al.,1986).  Basically,  cyclic  controls  are  periodic  variations  (zero- 
mean)  in  the  flows  between  components  in  an  ecosystem  or  between  the  ecosystem  and  the  surrounding 
environment.  If  the  amplitudes  and  frequencies  of  these  variations  ate  within  the  appropriate  range,  the 
ecosystem,  unstable  without  such  variations,  could  under  certain  conditions  be  stabilized  by  their 
introduction  without  anv  information  flows. 

Oscillations-induoed  stabilization  of  ecosystems  has  been  investigated  by  a  number  of  researchers. 
Armstrong  and  McOebee  (1SI76)  developed  a  theory  for  the  coexistence  of  a  variety  of  species  using  a 
smaller  number  of  resources.  Their  technique  involved  a  the  sequential  staging  of  the  species  in  a  periodic 
manner,  sharing  the  resource  through  time.  Kemp  and  Mitsch  (1979)  used  an  empirical  model  to 
demonstrate  the  stable  coexistence  of  three  plankton  species  on  the  same  resource  if  one  of  the  resource 
inputs  (wave  energy)  was  regularly  pulsing.  They  speculated  that  only  a  special  range  of  frequencies  and 
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pulse  amplitudes  vrauld  produce  the  needed  stability.  The  pulsing  resource  appeared  to  force  a  sharing 
between  the  three  species,  disadvanti^g  die  species  uiiich  was  the  most  prolific  under  steady  conditions. 
Levins  (1979)  established  the  sidfident  conditionS'trf  coexistence  by  requiring  that  the  resource  or  the 
species  functions  contain  externally  induc^  dme-varying  elements  that  enter  the  equations  nonlineariy. 
Nonlinear  dynanucs  in  Levins’  treatment  was  essential  since  it  residted  in  terms  with  even  powers  of  zero 
mean  oscillatory  functions.  The  averages  of  such  tenns  gave  rise  to  the  "average"  nonzero  inputs  which 
acted  as  effective  new  resources  and  under  cenain  conditions  ensured  stable  oscillatory  regimes  of  the 
system. 

The  goal  of  the  present  paper  is  to  assess  cyclic  (vibrational)  control  theory  as  a  totti  in  ecosystem 
analysis  and  management.  We  show  that  an  unstable  linear  system  can  be  made  asymptotically  stable  by 
zero  mean  paramenric  excitations  as  well,  and  hence,  nonlinearities  are  not  necessary  for  oscillatory 
stabilization.  We  also  utilize  nonzero  averages  of  even  powers  of  zero  mean  oscillatory  functions  to  obtain 
stabilizing  corrections.  However,  we  average  not  the  original  system  with  oscillations,  but  some  other 
specially  constructed  system,  the  average  of  which  reveals  the  dynamics  of  the  original  cycling  system. 
For  the  purpose  of  illustration,  we  have  chosen  a  modeling  technique  known  as  flow  analysis  (Hannon, 
1973,  1985a;  Barber,  et  al.,  1979)  firom  a  variety  of  ecosystem  modeling  approaches,  each  valid  for 
certain  system  classes.  First,  we  briefly  review  the  flow  analysis  technique  and  present  the  theory  of  linear 
cyclic  control  of  ecosystems.  Then,  we  apply  cyclic  control  to  an  oyster  reef  ecosystem  where  it  acts  in 
only  one  of  the  component  flows.  The  extension  of  the  theory  to  nonlinear  systems  can  be  done  on  the 
basis  of  the  work  of  Bellman,  et  al.,  1986.  The  theory  indicates  the  range  of  the  amplitude/frequency 
ratios  in  which  stalnli^g  cycles  should  be  sought  and  asserts  the  existence  of  stabilizing  eye'es  in  this 
range.  The  actual  stabilizing  anqilitudes  and  frequencies  are  detemiined  via  trial  and  error  solutions  of  the 
differential  equations. 

II.  FLOW  accounting 

In  the  analysis  of  complex  dynamic  systems,  it  is  necessary  to  develop  consistent  definitions  and 
categorize  all  the  identifiable  flows.  We  stan  with  the  diagram  shown  in  Figure  1.  For  more  details  on  the 
ecological  accounting  system,  see  Hannon  (1973),  Finn  (1976),  Levine  (1977, 1980),  Hannon  (1979), 
Patten,  et  al.,  (1976),  Herendeen  (1981),  Ulanowicz  (1984)  and  Hannon  (198Sa). 

In  Figure  1,  n  x  n  matrix  P  is  called  the  production-consumption  matrix^  This  matrix  represents 
n  processes  which  consume  and  produce  n  commodities.  By  process,  we  mean  an  aggregation  of  similar 
consumers-producers  which  is  viewed  as  a  single  ecosystem  component  By  commodities,  we  mean  the 
substances  produced  and  consumed  by  the  components  of  the  ecosystem.  The  elements  of  the  i^^  column 
represent  the  breakdown  of  the  main  pan  of  the  consumption  of  the  i*^  process.  The  elements  of  the  i^l^ 
row  describe  the  breakdown  of  the  main  pan  of  the  production  by  the  same  process.  Therefore,  each 

element  of  P  is  the  amount  of  commodity  i  (row  number)  which  is  used  by  process  j  (column  niunber)  in 
the  given  time  period.  For  example,  py  could  be  the  daily  amount  of  algal  biomass  (commodity  i) 

consumed  by  a  particular  class  of  herbivores  (process  j).  This  is  a  multicommodity  system  since 
commodities  listed  along  any  of  the  rows  are  noocommensurable  with  commodities  in  any  other  row. 
Therefore,  the  row  sums  may  be  calculated  since  they  are  all  the  tame  commodity  and,  we  assume, 
possess  the  same  nutritional  qualities  for  all  consumers  (The  exception  to  this  rule  is  the  nonbasal  heat  of 
respiration  which  by  definition  has  zero  value  to  any  component  in  the  ecosystem).  But,  in  general,  the 
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column  sums  cannot  be  fonoed  because  a  common  measure  of  a  value  of  each  element  along  the  columns 
may  not  exist  Commodities  of  differem  qualities,  even  though  measured  in  the  same  units  (e.g.,gms- 
carbon)  cannot  be  meaningfully  added  together.  The  inpuu  to  omnivores  and  detritivores,  for  example, 
are  of  different  qualities,  both  cfaemicaUy  and  in  nutritional  meaning,  to  the  consumer. 

The  diagonal  elements  in  P  «e  the  self-use  terms  which  are  for  example,  own-waste  consumption 
by  rabtxts  and  the  consumption  of  decomposers  by  decomposers  and  cannibalism. 

The  full  output  vector  q’  is  the  sum  of  the  vector  of  the  nonbasal  heat  w  given  off  by  each  of  tl>e 
components  and  the  total  output  vector  q. 

The  system  in  Figure  i  is  shown  without  joint  products,  that  is,  each  process  (column)  is  assumed 
to  produce  a  commodity  of  only  one  type.  The  joint  product  case  is  discussed  in  Hannon,  198Sa  and 
Costanza  and  Hannon,  1986. 

The  leladonship  to  the  external  environment  of  the  measurable  quanddes  in  the  ecosystem  modeled 
in  Figure  1  is  summarized  in  Table  1.  The  features  of  each  quantity  in  this  table  are  identified  by  the  letters 
in  the  corresponding  boxes.  The  table  shows  two  vectors:  r  and  a.  The  net  output  vector  r  is  composed 
of  three  types  of  flows:  exports  (A  &  D),  imports  (D  &  E)  and  the  heat  of  basal  metabolism  (B).  By 
imports  we  mean  those  quanddes  which  can  be  produced  by  the  ecosystem  but  enter  the  system  from  the 
external  environment.  Exports  are  those  quanddes  which  can  be  produced  by  the  ecosystem  but  which  are 
not  necessarily  produced  by  it,  and  which  leave  the  ecosystem  for  the  external  environment.  The  letter  D 
in  the  import  and  export  columns  indicates  those  measured  quanddes  which  are  passing  through  the 
ecosystem  in  the  given  dme  period,  thetefore,  the  quaitdty  A  -  E  is  the  net  export.  The  system  is 
penurbed  by  the  externally  induced  change  of  the  net  export.  The  heat  of  basal  metabolism  (basal 
respiradon)  is  that  given  off  by  the  organism  at  rest.  We  take  the  heat  of  basal  metabolism  (B)  as  a 
surrogate  for  the  commodity  flows  which  are  used  in  rebuilding  the  stocks  metabolized  during  the  given 
period.  By  stocks  we  mean  the  accumulated  output  quanddes  in  each  of  the  components  in  the  system. 


Net  Output 


W 


Figure  1.  Sieacfy  State  Ecosystem  Flow  Accounting  Diagram 


^Matrices  are  uppCTcase  symbols  and  vectors  are  lowercase;  both  ate  in  bold  type.  The  elements  of  either 
are  in  plain  type  with  the  appropriate  subscripdng.  A  dot  over  a  symbol  indicates  the  dme  derivadve. 
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Table  I.  Oescripcion  of  the  Quantities  which  Fotm  the  Net  Output  and 
Ndnpioduced  Input 

The  stocks  aie,  for  example,  the  amount  of  biomass  of  algae  which  has  accumulated  in  the  producer  (sun 
capturing)  component  of  an  aquatic  ecosystem.  The  vector  e  stands  for  those  input  commodities  that  the 
ecosystem  is  incapable  of  producing  (e.g.,  sunlight)  but  that  are  necessary  for  ecosystem  functioning. 

III.  FLOW  ANALYSIS 

Next  we  combine  the  flow  definitions  above  with  the  possibility  of  a  growth  in  the  stock  of 
process  j  during  the  given  time  period  At  These  flows  are  graphically  shown  in  Figure  2  for  the 
.  individual  process. 

The  consumption  flows  py,  production  flows  pj^  and  the  storage  flow  ASj/At  are  internal  to  the 
ecosystem  boundary,  while  the  net  output  flows  rj,  the  nonbasal  respiration  flow  wj  and  the  nonproduced 
input  flow  ej  cross  the  ecosystem  boundary.  The  nonbasal  respiration  flow  (c.  g.,  the  energy  used  in 
chasing  prey,  avoiding  predators,  food-searching  and  reproduction)  is  of  such  low  quality  that  it  cannot  be 
utilized  further  by  the  ecosystem,  and  it  is  therefore  considered  a  waste.  The  rj  consists  of  the  net  expon 
of  the  process  (export  minus  import)  and  the  stock  replacement  (basal  respiration).  The  net  input  vector  e 
is  assumed  to  cause  no  restriction  to  the  level  of  qj  and  is  dropped  from  further  consideration  at  the  current 
stage  of  the  model  development 

The  total  outflow  q  j  is  defined  for  the  steady  state  ecosystem  as 

q'j- JPjk  +  rj +Wi  .  (1) 

To  take  into  account  a  growth  in  stock,  Asj,  over  the  time  period  At  when  the  system  is  not  in  the  steady 
state,  definition  (1)  is  augmented  as 
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Figure  2.  The  Definition  of  the  Input  and  Output  Rows  of  a  Typical  Process  (j) . 


Three  important  simplifying  assumptions  are  now  made  for  the  ecosystem  shown  in  Figure  2  with 
qj'  defined  in  (2). 

i)  a  commodity  weighting  or  importance  factor  is  assigned  to  each  of  the  commodities  produced  in 
the  system.  The  weight  for  each  commodity  is  independent  of  which  component  consumes  this 
commodity.  A  weight  of  zero  is  given  to  the  nonbasal  heat  of  respiration,  and  therefore,  the  vector  w 
disappears  fwm  the  formulation.  The  element  qj  can  be  then  be  fonmed  by  the  simple  addition  of  all  the 
elements  along  the  j***  row  of  matrix  P,  the  rate  of  the  j***  stock  growth  and  the  j***  element  of  vector  r.  For 
a  more  complete  discussion  of  the  commodity  weighting  issue,  see  Hannon,  1985a. 

ii)  the  inputs  to  process  j,  py  form  a  constant  ratio  with  the  output  of  process  j,  qj.  Thus,  p^j/qj  = 
gjj  s  constant.  The  constants  gjj  are  determined  from  the  data  on  the  ecosystem  at  its  steady  state  and 
they  are  assumed  to  remain  constant  for  the  dynamic  form  of  our  model  presented  below.  These  constants 
represent  the  internal  behavior  of  the  process.  The  gjj  incorporate  the  consumption  flows  into  the 
model  by  locking  them  into  a  constant  relationship  with  the  output  of  the  receiving  process.  Thus,  the 
problem  of  summing  the  consumption  flows  (see  Rgure  2)  is  avoided. 

iii)  the  stock  (sp  of  any  process  O')  says  in  constant  proportion  to  the  total  output  (qj)  of  this 

process.  That  is;  b^j  =  Sj/qj  ■  constant,  forming  a  diagonal  matrix  B  =  diaglbu . bnn).  This 

assumpdon  allows  us  to  obtain  a  balance  equation  using  definidon  (2)  since  now 

=  .  (3) 

If  the  results  of  assumptions  i)  and  ii)  are  combined  with  (2)  and  (3),  and  if  At  becomes 
infinitesimal,  we  have 


—  =  Zg  T^+  r  +  S. 


(4) 


where  l*d^/dt. 
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Equation  (4)  is  the  dynamic  description  of  the  stock  for  process  j.  However,  most  experimental 
ecosystem  data  is  presented  as  flows.  Therefore,  we  change  (4)  into  a  dynamic  description  of  the  flows 
for  process  j.  Substituting  (3)  and  its  lime  derivative  into  (4)  yields 

=  vj.isjsn. 

or  in  matrix  foim 

q  =  Aq  -  B  r,  A  4  B  ‘a  ■  G).  (5) 

This  time  invariant  ordinary  differential  equation  (5)  is  in  the  "standard"  form  for  the  flow  analysis 
approach. 

IV.  STABILITY  ANALYSIS 

The  stability  properties  of  the  behavior  of  q  when  the  system  is  subjected  to  a  step  change  in  r 
depend  entirely  on  the  matrix  A  in  (S).  If  the  real  parts  of  all  the  eigenvalues  of  A  were  negative,  the 
system  would  respond  in  a  stable  manner  (Luenberger.  1979,  p  158).  However,  in  (5)  the  sum  of  the 
eigenvalues  of  matrix  A  is  always  positive.  Therefore,  the  system  will  always  respond  to  ’'sufficiently 
rich"  changes  in  r  in  an  unstable  manner. 

From  an  ecological  viewpoint,  a  positive  r  represents  an  output  of  the  ecosystem  (for 
example,  the  amount  of  fish  caught  in  the  annua!  season).  From  the  control  theory  viewpoint  however, 
this  output  represents  an  input  to  the  system  or  a  control  action.  For  example,  the  amount  of  fish  caught 
directly  affects  the  rate  of  (re)production  of  fish  and  many  other  quantities  produced  in  the  ecosystem, 
which  in  turn,  also  affect  the  fishing  success.  If  the  system  (5)  is  to  accurately  represent  the  functioning  of 
an  ecosystem,  the  equations  must  be  judiciously  modified  to  include  stabilizing  or  controlling  flows. 
Equations  (5)  can  be  made  to  respond  stably  by  modifying  r  to  include  a  feedforward  or  a  feedback 
control.  Let  us,  however,  demonstrate  the  use  of  cyclic  control  for  ecosystem  stabilization  through  the 
addition  of  a  cyclic  flow  to  one  of  the  elements  in  the  matrix  G. 

In  the  flow  accounting  framework,  cyclic  control  alone  cannot  guarantee  stability  of  the  system. 
However,  only  a  veiy  simple  form  of  constant  feedback  is  required' to  make  cyclic  control  effective.  Such 
feedback  can  be  easy  to  maintain  since  it  need  not  ensure  stability  but  only  "condition"  the  system  for 
cyclic  control.  On  the  other  hand,  for  a  broad  class  of  the  so-called  decentralized  systems,  no  constant  or 
time-varying  feedback  exists  that  can  stabilize  the  system  (Anderson  and  Moore,  1981).  In  these  cases, 
the  addition  of  cyclic  control  can  result  in  the  desired  stabilizing  effect 

Since  equation  (5)  is  still  always  unstable,  several  changes  must  be  made  to  r  to  demonstrate  the 
cyclic  control.  First,  r  must  be  broken  into  two  parts:  a  vector  of  net  outputs  which  are  independent  of  the 

output  q,  and  another  vector  which  contains  the  feedback  and  cyclic  control  and  depends  on  q.  The  first 
vector  contains  the  "set  point"  vector  for  the  system,  r^t  the  vector  of  net  outputs  which  in  the  absence  of 

cyclic  control  determines  the  unstable  steady  state  level  q^  of  the  total  output.  The  introduction  of  cyclic 
control  converts  the  unstaUe  steady  state  q^  into  asymptotically  suble  T-periodic  operating  regime,  q^(t), 

where  T  is  the  period  of  a  cyclic  control.  A  feedback  control  is  needed  to  convert  the  trace  of  the  matrix  in 
equation  (5)  to  a  negative  value  (Meerkov,  1980).  Assume  that  this  is  an  internal  control  that  changes  the 
net  output  from  the  system  in  linear  proportion  to  the  production  flows,  a  "flow"  control  (Haimon,  1986). 
For  simplicity,  let  the  linear  proportionality  be  represented  by  a  diagonal  matrix  of  constants,  Q.  In  this 
case,  vector  r  in  equation  (5)  is  given  by: 


/ 
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r  =  +  Fj  =  r,  -  r,  +  Qq  , 

where  Tj.  =  Q<^  ()  ) 

Equation  (5)  then  becomes: 

5  =  b  Vi  -  G  -  Q)q +B  (r.- 
=  Nq  +B  ‘(r^  -  r,),  N  =  B'‘(I  -  G  -  Q).  (6) 

The  constant  vector  will  be  dropped  because  it  is  independent  of  q  and  therefore  docs  not 

affect  the  stability  analysis. 

Matrix  Q  must  have  only  one  non-zero  element  with  sufficiently  large  absolute  value  to  cause  u 
sign  change  in  the  trace  of  N.  Therefore,  we  funher  assume  that  matrix  Q  makes  the  trace  of  manix  N 
negative,  but  does  not  guarantee  system  stability,  i.  e.,  we  simulate  the  circumstances  where  the  feedback 
controls  Oike  Q)  are  not  adequate  to  make  all  of  the  eigenvalues  fall  in  the  left-half  plane.  This  situation 
can  arise  if  the  information  gathering  processes  of  the  system  are  somehow  limited,  resulting  in  lack  of 
controllability  and/or  observability  (Luenberger,  1979),  but  are  sufficient  to  condition  the  system  for  cyclic 
control. 

Let  us  again  augment  the  vector  r  =  Qq  -  D(t)q,  where  D(t)  is  a  periodic,  zero  mean  matrix.  The 
periodic  input  D(t),  is  weighted  by  the  state  vector  of  the  system  q,  and  therefore  D(t)  appears  in  the 
system  equation  in  the  form  of  parametric  penurbations  or  cyclic  control.  In  this  case  equation  (6) 
becomes 

q  =  [N  +  B'^DCOlq  .  (7) 

Because  equation  (7)  is  time-varying,  eigenvalues  can  no  longer  describe  its  stability.  It  is 
possible,  however,  to  associate  stability  properties  of  the  oscillatory  system  (7)  with  a  certain  constant 
matrix  that  describes  its  average  behavior.  The  stabilizing  action  of  cyclic  controls  consists  in  converting 
the  remaining  right-half  plane  eigenvalues  of  system  (6)  into  "left-half  plane  on-the-average"  ones.  In  this 
case,  stabilization  is  achievable  without  the  need  for  additional  informabon  flows,  provided  that  the 
amplitudes  and  frequencies  of  the  cyclic  controls  are  within  a  cribcal  range. 

Assume,  for  simplicity,  that  the  ij'^  element  of  the  cyclic  control  matrix  D(t)  is  given  by  dj/t)  = 
CijCOs((i>..t),  where  c^j  is  the  amplitude  and  oiij  is  the  frequency  of  the  oscillabon. 

In  order  to  describe  the  average  behavior  of  system  (7),  we  introduce  the  parameter  E  as 

e  =  max(l/  cojj ) 
ij  y 

and  define 

Cjj  =  ttij/e  and  (0  =  |3  ij/e 

so  that  the  ij***  element  of  D(t)  can  be  rewrinen  as  d.^(t)  ■  (a /e)cos(P.  Ve). 

With  this  notabon,  the  cyclic  control  matrix  D(t)  takes  the  form 
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D(t)  =-^D'(X), 

and  system  (7)  becomes 

4  =  [N  +  (8, 

Thus,  if  the  a  .'s  and  S.  's  are  assumed  constant,  the  amplitudes  c..  and  the  frequencies  (o..  of  the 

ij  <j  >j  ‘j 

zero-mean  cyclic  terms  d,^(t)  are  parameterized  by  a  positive  e.  It  has  been  proven  (Bellman,  Bentsman 
and  Meerkov,  1985)  that  there  exists  an  =  constant  >  0,  such  that  for  any  e  satisfying  the  inequality  0  < 
e  <  ,  the  stability  piopenies  of  system  (8)  are  defined  by  the  eigenvalues  of  a  constant  matrix 


M  =  lim—  <!>('$  dx 


where  Oft)  is  the  state  transition  matrix  of 


^=BV(r)q 


(9) 


(10) 


where  t  =  i/e. 

Specifically,  for  sufficiently  small  e,  system  (8)  is  asymptotically  stable  if  all  the  eigenvalues  of  M  have 
negative  real  parts.  As  seen  from  this  result,  the  elements  of  matrix  M  are  defined  in  terms  of  the  elemenis 
of  matrices  N,  B*^,  "amplitude/frequency"  ratios  a..,  and  ''frequency/frequency"  ratios 
Consequently,  M  provides  a  link  between  a.^,  p^^  and  stability  of  (7);  If  a.^  and  p,^  are  found  which  place 
all  the  eigenvalues  of  M  in  the  left-half  plane,  then  there  exists  an  e  such  that  oscillations  with  amplitudes 
a  jt  and  frequencies  p /e  guarantee  asymptotic  stability  of  system  (7).  The  matrix 

M'  ^  M-N 


can  be  thought  of  as  a  "correction"  of  N  induced  by  oscillations. 

In  the  context  of  ecological  systems,  cyclic  control  is  easy  to  apply.  Indeed,  ecological  systems  are 
usually  described  by  sparse  matrices  and  therefore  the  cyclic  control  matrix  D(t)  might  often  satisfy 
condition  D'ft)  =0  independently  of  the  magnitudes  and  frequencies  of  the  oscillations.  In  this  case, 
since  B  is  a  diagonal  maoix,  all  non-zero  elements  of  matrix  M'  are  given  as 


Y 

•  ii  "  T* 


aii 

bii  P, 


(11) 


where  n  ^  denotes  the  ji'*’  element  of  the  matrix  N.  Therefore,  the  only  elements  of  D(t)  that  will  affect  the 

eigenvalues  of  M  are  those  off-diagonal  dements  that  have  a  corresponding  non-zero  symmetric  elc^int  in 
N. 


The  first  step  in  the  search  for  amplitudes  and  frequencies  of  the  stabilizing  oscillations  is  to  find 
m'.^'s  that  move  all  the  eigenvalues  of  N-t-M'  to  the  left-half  plane.  A  straightforward  way  to  accomplish 
this  IS  to  try  only  one  of  the  appropriate  elements  at  a  time,  and  let  g.  increase  from  0  to  a  sufficiently 
large  number.  When  the  appropriate  set  of  elements  m'.^,  and,  hence,  g..  have  been  identified,  we  must 
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Table  2.  Oyster  reef  Input-Output  flow  matrix  (P),  along  with  vectors  for  net  export  +  stock 
replacement  respiration  (r),  total  output  excluding  waste  heat  (q),  waste  heat  (w), 
aixl  total  output  including  waste  heat  (q*). 


P 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

r 

4 

w 

4' 

Oysters  1 

0 

15.79 

0 

0 

0 

0.51 

17.80 

34.10 

7.365 

41.47 

Detritus  2 

0 

0 

8.17 

'  7.27 

0.64 

0 

6.19 

22.27 

0 

22.27 

Microbiota  3 

0 

0 

0 

1.21 

1.21 

0 

2.875 

5.295 

2.875 

8.17 

Meiofauna  4 

0 

4.24 

0 

0 

0.66 

0 

1.75 

6.65 

1.75 

8.4 

Deposit  Feeders 

S  0 

1.91 

0 

0 

0 

0.17 

0.215 

2.295 

0.215 

2.5! 

Predators  6 

0 

0.33 

0 

0 

0 

0 

0.2 

0  53 

0.15 

0.68 

Net  Input  e 

41.47 

0 

0 

0 

0 

0 

Control  Q 

1.52 

2.28 

.94 

1.26 

2.09 

1.38 

return  to  equation  (8),  placing  (a7e)cos(p,/e)  at  these  locations  in  D(t).  Then,  by  changing  e  and 
repeatedly  solving  equation  (8)  for  stabilizing  pairs  of  (a,P).  the  areas  of  stabilizing  amplitudes  and 
frequencies  can  be  found.  The  search  for  stabilizing  oscillations  becomes  complicated  when  the  stabilizing 
matrices  that  satisfy  D^(t)  =  0  do  not  exist  (see  for  example,  Wu,  1975). 

Cyclic  control  could  naturally  arise  in  an  ecosystem  as  i)  an  oscillation  of  the  flows  between 
various  components  or  ii)  a  part  of  the  net  output,  a  cyclical  export  (import)  from  (to)  a  particular 
component,  the  interpretation  used  in  this  paper. 

What  follows  is  a  simple  example  of  ecos>  tern  stabilization  by  a  cyclic  control. 

V.  application  TO  THE  OYSTER  REEF  ECOSYSTEM 

In  this  section,  we  apply  the  theory  presented  above  to  the  oyster  reef  ecosystem  (Dame  and 
Patten,  1981).  This  compact  but  complex  system  is  shown  at  steady  state  ( i.e.,  for  constant  flows)  in 
Figure  3. 

The  data  from  Figure  3  have  been  arranged  in  the  proposed  accounting  framework  (Figure  1)  in 
Table  2.  In  this  arrangement,  estimates  of  the  basal  metabolism  or  structt-ai-rebuilding  respiration  are 
included  in  the  net  output. 

From  the  data  in  Table  2,  we  constructed  G  for  use  in  the  N  matrix.  With  the  feedback  control 
elements  of  diagonal  matrix  Q,  shown  in  Table  2,  the  trace  of  N  is  negative  and  its  eigenvalues  are: 
0.0726  ±0.037  li,  -0.1753,  -.0089,  -0.0994  and -0.(X)28.  Because  the  complex  pair  has  positive  real 
parts,  the  system  is  unstable.  Let  us  demonstrate  that  a  cyclic  control  can  be  found  to  stabilize  the  system 
at  the  given  steady  state. 
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Figure  3.  The  Oyster  Reef  Ecosystem.  Row  units  are  kcal/m  -day.  Stock  unit  kcal/m . 


Let  ^  be  the  only  non-zero  element  of  matrix  M’,  indicating  a  cyclic  net  input  to  deposit 
feeders  and  a  cycle  in  the  flow  5-3.  Then  by  experiment,  for  m'^^  >  0.0346,  all  the  eigenvalues  of  matrix 
N  +  M'  are  in  the  left-half  plane.  Choosing  ==  1.0,  from  equation  1 1  we  obtain 


1.7298, 


where  b^  7.0893  and  njj*-1.1632.  Thus,  according  to  the  theory  of  Section  IV,  oscillations  of  the 
form  dj  ^(t)  *  ajjtnSinftot),  >  1.73,  should  stabilize  the  system  for  sufficiently  large  to.  The 
asymptodc  nature  of  the  theory  implies  however,  that  condition  >1.73  should  be  partially  observed 
for  smaller  O)  as  well.  It  is  precisely  this  insight  that  motivates  the  numerical  search  for  the  actual 
parameters  of  stabilizing  cycles  at  low  hequencies.  In  Figure  4,  we  demonstrate  that  condition,  ^ 
>1 .73,  is  partially  observed  ftw  tij/2jt  >  0.08.  The  amplitudes  are  Qj^jjftl/bj^.  The  cross-hatched  region 
in  Figure  4  corresponds  to  the  actual  stabilizing  amplitudes  and  frequencies  of  the  cycles  ^(t). 
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Frequency,  Cycles/day 


Figure  4.  Cyclic  Control  in  the  Oyster  Reef  Ecosystem.  The  Range  of  the  Parameters  of  the 
Stabilizing  Oscillations  of  the  Net  Input  to  the  Deposit  Feeders  S  and  of  the  Connection 
to  the  Microbiota  3. 

While  our  choice  of  Q  was  largely  arbitrary,  we  find  the  data  in  Figure  4  interesting.  They  show, 
for  example,  that  a  cyclic  net  input  to  the  Deposit  Feeders  (which  in  turn  allows  them  to  cycle  their  feeding 
on  the  microbiota)  can  stabilize  this  ecosystem  (given  the  above  (J).  With  a  cycle  fiequency  of  once  in 
seven  days,  the  stabilizing  amplitude  would  range  from  about  1.1  to  1.7  kcals/m  -day,  encompassing  the 
average  value  of  the  flow  from  3  to  5  of  1.2  kcals/m^-day  (see  Figure  3).  It  seems  possible  that  such  a 
cyclic  flow  could  occur.  No  data  on  the  variation  of  flows  in  this  oyster  reef  ecosystem  were  given 
(Dame,  1976,  1979;  Dame  and  Patten,  1981).  From  Figure  4,  we  also  see  that  smaller  stabilizing 
amplitudes  are  associated  with  lower  frequencies.  This  application  to  the  oyster  reef  system  is  expected  to 
convey  a  biological  possibility  of  ecosystem  stabilization  by  already  existing  or  intentionally  introduced 
oscillations. 

VI.  CONCLUSION 

The  material  presented  above  demonstrates  that  cyclic  control  is  a  biologically  feasible  stabilizing 
mechanism  that  could  either  develop  in  the  course  of  evoludon  or  be  introduced  by  an  ecosystem  manager. 

The  important  point  about  cyclic  control  is  that  stabilization  can  be  provided  without  any 
informanon  exchange.  Therefore,  the  components  that  can  establish  a  balatKcd  cyclic  exchange  of 
materials  or  energy  with  the  external  environment  aixVor  with  other  components  might  bring  stability  to  the 
whole  system  without  the  cost  of  building  and  maintaining  additonal  informanon  links.  Thus,  since  cycles 
often  occur  in  ecosystems  naturally  or  can  be  introduced  iniendonally,  cyclic  control  theory  constitutes  a 
viable  tool  for  the  ecosystem  analysis  and  management. 
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SELF  CONTROLLED  GROWTH  POLICY 


FOR  A  FOOD  CHAIN  SYSTEM 
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Canada 


Abetvaat.  A  behavioural  policy  of  controlled  growth  for  a  food  chain 
model  of  length  2n  is  considered.  The  highest  trophic  level  popula¬ 
tion  controls  its  own  growth  in  order  to  restrain  the  growth  of  the  o- 
ther  2n-l  populations  in  the  system  so  as  to  avoid  undesirable  out¬ 
comes  . 


1.  INTRODUCTION 

The  present  research  concerning  control  policies  for  biological  systems 
in  population  dynau-ilcs  mainly  deals  with  human  control  added  to  models 
of  interacting  populations.  Various  pest  management  programs  provide 
typical  examples  of  this  kind  of  external  control  (1,2].  However  in  re¬ 
ality  there  are  also  situations  in  which  one  or  more  populations  partic¬ 
ipating  in  the  system  are  the  controllers.  Such  systems  change  behav¬ 
iour  abruptly  in  response  to  changes  of  the  size  of  the  interacting  pop¬ 
ulations,  climatic  conditions,  diseases,  etc.  We  call  this  type  of  con¬ 
trol  internal.  The  classical  models  in  population  dynamics  usually  do 
not  reflect  either  the  external  nor  the  internal  control.  The  control¬ 
ling  populations  can  apply  the  internal  control  to  their  own  members 
{self  control)  or  to  all  or  some  of  the  other  participating  populations  in 
the  model.  In  this  paper  the  attention  is  focused  on  the  concept  of 
self  control. 

Generalizing  a  previous  paper  (Bojadziev  and  Skowronski  (3))  here  we 
study  a  food  chain  system  of  size  2n  involving  a  controlling  factor 
u(t)  which  adjusts  the  number  of  the  highest  trophic  level  population 
so  that  a  reasonable  size  of  all  populations  is  maintained.  Making  use 
of  a  methodology  developed  by  Leitmann  and  Skowronski  [4]  (see  also 
Blaguiere,  Gerard,  and  Leitmann  [5])  for  dynamical  systems,  we  derive 
conditions  under  which  the  designed  control  policy  results  in  avoidance 
of  a  prescribed  region  in  R^”  so  that  undesirable  outcomes  are  avoided. 

2.  THE  FOOD  CHAIN  MODEL 
Consider  the  food  chain  model  with  control 
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5’(t)  -  f(x(t),  u(t))  (1) 

—  T 

where  t  €  is  the  time  variable,  x(t)  «  (Xj^, . . .  zXjjj)  is  the  popu¬ 
lation  vector,  u(t)  is  the  control,  and  the  components  of  the  vector 
function  f(x,u)  =  (f  . . .  ,f2^)  are  given  by 

6, 

fj^(x,u)  =  *2^  ' 


f2k 


'■2k 
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f.n(x,u)  =  x^^ 
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+  2n-l  ^ 

2n  Y2„  ^2n 


-4 


+  ux 


2n 


k=l,...,n-l,  f^(x,u)  =  f^(x,0),  i=l,...,2n-l  . 

For  u=0  the  model  (1)  reduces  to  the  uncontrolled  food  chain  model 

X'  (t)  =  f  (x(t)  ,0)  .  (3) 

In  (1)  x^,i=l, . . . ,2n,  is  the  size  of  the  i-th  population;  (growth 

rate  coefficient),  (interaction  coefficient),  and  y^  (trophic  weight 

factor)  are  positive  constants;  y/y-  expresses  the  gain-loss  ratio 

3  ^  ^ 

when  population  i  interacts  with  population  j.  The  control  u(t)  €  U[t^j,t) 

lu(t):  u(t)  i  U  and  u(t)  measurable  on  [t^,t)),  0<t  <t<“,  UCR 

o  o 

is  a  compact  set  to  be  specified  later  in  accordance  to  a  growth  restric¬ 
tion  policy. 

2 

The  biological  meaning  of  the  control  term  <^20  last  expression 

(2)  which  takes  part  in  (1)  is  that  for  u  >  0  tite  population  with  size 
X2jj  (the  highest  trophic  level  population  in  the  food  chain)  is  enhanced 
by  increasing  the  population  density  (Increasing  returns)  2md  for  u  <  0 
it  dampers  its  own  growth  (diminishing  returns) .  The  2n-th  population 
C2m  be  considered  as  a  consumer  or  predator  of  a  higher  level  in  terms 
of  organization  and  brain  capability  in  comparison  to  the  other  2n-l 
populations  or  resources.  The  self  controlled  growth  of  the  consumer 
(predator)  will  affect  the  growth  of  all  populations  in  the  food  chain 
system. 

Each  choice  of  control,  say  u(tQ)  •>  c^j  €  U  on  some  time  interval  start- 


-- 
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ing  at  t  =  t^,  generates  a  solution  or  response  kit]  ■  klilt^),  c^,  t) 
o£  the  system  (1)  with  initial  state  x(t^)  €  which  geometrically 

is  represented  by  aui  orbit  in  the  phase  space  If  ■  0 

(no  control,  hence  (1)  reduces  to  (3))  the  response  k(x(t^),  0,  t)  of 
(3)  can  exhibit  large  variation  and  may  endanger  the  existence  of  an 
acceptable  size  of  some  populations.  In  order  to  avoid  such  undesirable 
outcomes,  the  consumer  population  with  size  Xj^^  may  opt  to  self  con¬ 
trol  its  own  growth  which  will  affect  the  growth  of  the  other  populations 
in  the  food  chain.  This  can  be  accomplished  by  selecting  a  suitcUsle 
control  value  u(tj^)  =  Cj^  €  U  at  a  point  x(tj^)  €  R^"  (switching 
point)  on  some  time  inverval  starting  at  t  =  t^^,  t^^  >  t^.  The  control 
value  u(tj^)  =  Cj^  will  generate  a  response  k(x{tj^),  c^,  t)  along  a 

new  orbit  ig  n  =  x(tj^). 

Using  a  Liapunov  function  for  the  uncontrolled  model  (3)  we  define  for 
the  response  of  (1)  an  avoidance  region  A,  a  security  zone  S  which 
safeguards  the  response  of  entering  A,  and  design  a  control  policy  for 
avoidance. 

3.  THE  LIAPUNOV  FUNCTION 

The  coordinates  of  the  nontrivial  equilibrium  E®(x®),  x*^  =  (xj , . . .  ,X2jj) 
i  of  (1)  are 

0  _  “l’''l  „0  _  “2n^2n 


“2k^2k  *  ®2k*2k+l 


k=l,...,n-2  , 


’‘2k+1^2k+l  ®2k’'2k 


k»l, . . . ,n-l 


We  require  that  E^  f  Int  R^”,  the  interior  of  the  closed  positive  cone, 

so  that  E°  has  biological  meeuiing.  Since  *2n-l  ^  follows  from 

(4)  that  X2k-1  ^  • • • '"“!•  Also  from  (4)  we-see  that  Xj  >  0. 

However,  in  order  to  secure  that  x?.  >  0,  k«2,...,n-l,  we  assume  that 
0 

*2k  “2k+l  ^2k+l/®2k’ 

The  model  (3)  has  the  Volterra  function  (Huang  and  Morowitz  (61) 


2n  n  /  * ■  \ 
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continuous  on  Int  which  is  actually  a  Liapunov  function  with  the 

following  properties. 

(i)  The  minimum  of  V(x)  is  attained  at  the  equilibrium  E^(x^)  given 
by  (4);  minV(x°)  =  0; 

(ii)  V(x)  is  monotone  increasing  about  (has  the  nesting  property) 

(6) 


au)  |^g-f,(x.O)  =0  . 


where  f^  are  given  by  (2) •  Prom  here  follows  that  the  equilibrium 


E*^(x^)  is  stable. 


The  model  (3)  has  a  first  integral 

V{x)  =  h,  h  =  const  >  0  ,  (7) 

which  represents  a  family  of  level  surfaces  V.  in  The  orthog- 

2ii  ^ 

onal  projection  of  U.  onto  R  generates  2n  dimensional  hypersur- 

On  ^ 

faces  H.  in  R  which  are  closed,  do  not  intersect,  contain  inside 

h  Q 

the  equilibrium  E  ,  and  accommodate  orbits  of  (1).  Further,  if  hj^  <  hj 
the  hypersurface  H.  is  inside  the  hypersurface  H  . 

4 .  AVOIDANCE  CONTROL 

Here,  marking  use  of  a  Liapunov  design  technique  [41 ,  we  introduce  def¬ 
initions  and  prove  a  theorem  concerning  the  food  chain  model  (1) . 

—  T  2n 

Definitton  1  (Avoidance  set  A).  Given  e  =  ^ 

and  the  Liapunov  function  V(x)  by  (5), 

A  A  {x  €  R^":  V(x)  >  V(e)  -  h^)  ,  (8) 

where  (avoidance  parameters),  i>l,...,2n,  are  small  as  desired  for 

a  particular  study.  The  boundary  of  A  is 

3A  =  H.  a  (x  €  R^":  V(x)  »  h  )  .  (9) 

h^  -  e 

T  T  2n 

Definition  2  (Security  zone  S)  .  Given  o  «  ^*1' •  •  • ''^2n^  ^  ' 

>  e^,  and  V(x)  by  (5), 

S  a  fx  €  R^":  V(x)  ■'  V(6)  -  hj )  -  A  ,  (10) 

5^,i»l, . . . ,2n,  are  security  par2uneters.  The  Ijoundary  of  S  is  given  by 

3S  -  H.  a  (x  €  R^”:  V(x)  ■  h.)  .  (11) 

h^  -  « 

Prom  the  nesting  property  of  V(x)  it  follows  that  hj  c  h^ ,  hence  in 
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the  hyper surf ace  (9)  encloses  the  hyper surface  (11) . 


Definition  3  The  set  A  defined  by  (8)  is  avoidable  if  there  is  a  set 
S  defined  by  (10)  and  a  control  u  €  0  such  that  for  all  x°(tg)  €  S, 
the  response  )c (x* (t^) ,u (t^) ,t)  of  (1)  cannot  enter  A,  i.e. 

lc(x®(t„) ,  u(t^)  ,  t)  n  A  =  *  V  t  .  (12) 
s  s 


Now  we  establish  sufficient  conditions  for  the  avoidance  of  A. 


Theorem  The  food  chain  model  (1)  is  controllable  for  avoid2moe  of  A 
if  there  is  a  control  u(t)  €  U  emd  a  Liapunov  function  V(x)  defined 
by  (5)  so  that 


dV(x) 

dt 


2n  3V 


f  (x,u) 


5  0  , 


(13) 


where  f^(x,u)  are  given  by  (2). 


Proof.  Assume  that  A  is  not  avoidable,  i.e.  (12)  is  violated.  Hence 
for  some  x®(tg)  i  S,  the  response  )c{x®(tg),  u(tg>,t)  enters  A,  t  >  t^. 

Then  there  is  a  t  >  t„  for  which  x^(t  )  «  lc(x®(t„) ,u(t  ) ,t.)  €  3A. 

as  a  s  s  a 

From  the  nesting  property  of  V(x)  it  follows  that  V(x®(tg))  <  V(x*(tg)), 

meaning  that  the  function  V(x)  is  increasing.  This  contradicts  (13) 

which  states  that  V(x)  is  non-increasing  along  every  response  of  (1). 

5.  THE  CONTROL  POLICY 

To  design  a  policy  for  avoidance  the  region  A  by  the  response  of  (1) 
we  use  the  theorem  in  the  previous  section.  Substituting  from 

(2)  into  (13)  with  (5)  gives 


av  2 

1  ■  -■  UX- 

5x^  2n 


According  to  (6)  the  summation  term  above 


f2n^ 


oil  1  \  2  ^ 


which  can  be  written  as 

(t-t)"- 


0  . 

is  zero;  the  second  term  gives 


(14) 


The  Inequality  (14)  establishes  a  relationship  between  the  control  u 
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and  the  controlling  population  It  requires  that 


o 

VI 

S3 

^  *2n  > 

*2n 

C 

IV 

o 

^  *2n  ^ 

*2n 

According  to  (15)  we  specify  that 

u(t)  €  U  =  [-r,r]  c  R,  r=con3t.  (16) 

I 

On  the  basis  of  (15)  we  formulate  the  following  behavioural  policy. 

Avoidance  control  policy:  If  the  response  )ttt]  =•  )c  (x  (t^)  ,u  (t^)  ,t)  of 
the  food  chain  model  (1)  with  initial  state  and  fixed  control 

u(t^)  €  0,  U  specified  by  (16)  ,  enters  the  security  zone  S  given  by 
(10),  in  order  to  prevent  )c[t]  of  entering  into  A  defined  by  (8),  a 
new  control  value  ii(tg)  should  be  selected  from  U  at  a  switching 
point  x(tg)  (  S  with  corresponding  response  lc(x(tg) ,u(tg) ,t) ,  t^  >  t^. 

If  the  new  control  value  tt(tg)  should  be  negative  and  if 

X2n  ^2n'  should  be  positive. 

Note  1.  The  control  u=0  satisfies  (15)  but  then  the  response  will  be 

accommodated  on  a  hypersurface  enclosed  in  the  security  zone  S, 

s 

Hh  <  ^  satisfactory  since  large  population 

S  "s  e 

fluctuations  occur. 

Note  2.  The  particular  situation  X2jj  “  at  x(tg)  €  S  satisfies 

(14) ,  hence  any  value  u  f  U  cem  be  selected  temporarily  until  the  re¬ 
sponse  moves  to  a  neighbouring  point  in  S  for  which  X2jj  ^  ’^^n* 
the  avoidance  control  policy  can  be  applied. 
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MATHEMATIQUES  ET  SYSTEMES,  ASPECT  CALCUL 


MATHEMATICS  AND  SYSTEMS.  COMPUTATIONAL  BEARINGS 


QDMILINUUUmiON  H  biological  systems  MODELIB6 


B.  S.  Ua*  and  t.  M.  Mans** 


Tba  aatiutloa  of  paraaatars  in  diffaraatlal  aqvatioaa  ia  a  basic  problaa  ia 
biological  flyttaaa  aodaliag.  Hovaaor,  thoaa  paraaatara  caaaot  ba  aatiaatad  aasily 
abac  tba  aquatioas  ara  too  coaplicatad  aad  caaaot  ba  tolvad  ia  elosad  fora. 

Altboagb  Sr.  Ballaaa  baa  propoaad  to  aaa  guasiliaaarizatioa  to  aolaa  tbia  problaa, 
aora  auaarical  azpariaaata  ara  aaadad  to  show  tba  affactiwaaess  of  this  approach, 
la  this  papar,  guasiliaaarizatioa  is  usad  to  astiaata  tba  paraaatars  ia  various 
biological  aodals.  It  is  showa  that  this  approach  is  quits  affaetiwa  sad  coavsrges 
wary  fast  ia  aost  situatioas.  Thus,  tba  quadratic  coawarganea  proparty  is 
prasarwsd. 


QUASILIMEARIZATIOM  AMD  THE  MOWLimtAB  ESTIMATTOM  OF  PASAlIKTKas 


The  algoritha  of  quasilinearizatioa  ia  astiaation  is  wall  docuaantad  [1-3] ,  oaly 
the  assential  aquatioas  will  ba  discussed  ia  the  followiag.  Coasidar  a  systea 
raprasantad  by  tba  followiag  systaa  of  aoaliaaar  diffarsatial  aquatioas 


If  -  f(x,  Q,  t) 


(1) 


whera  x  aad  f  ara  M-diaaasioBal  vectors  with  coapoaaats  x^,  X2,  X||  aad  f^,  f2> 

...,  f||,  respectively  aad  a  represaats  the  L  diaaasioaal  uakaown  paraaatars.  Let  us 
assuaa  that  the  L  paraaatars  caaaot  ba  aaasurad  directly  aad  oaly  of  tba  M 
variables  caa  ba  aaasurad.  These  aaasurad  valuas  ara 

Xj‘**’*’{t,)  -  S  -  l,3,...,a,  j  ■  l,2,,.,,Mj  (2) 

with  problaa  is  to  astiaata  tba  paraaatars  /  >  1,2,...,L  aad 

tbs  iaitial  conditioas 

Xi(0)  ■  Cj,  i  ■  1,2, ...,M  (3) 

froB  tba  givsn  or  aaasurad  data,  Equatioa  (2).  It  should  bo  oapbasizod  that  tba 


aaasurod  valuas  b^^^^  do  coataia  aoisa.  Lot  us  astablisb  tba  vactor  aqaatioa 


(4) 


•  Corraspoadiag  author,  E.  S.  Laa,  Oapt.  of  lad.  Eagg.,  Eaasas  Stats  Oaivorsity, 
Maahattaa,  E8  6650$ 

**  Dopt.  of  lad.  Eagg.,  Tsiagbua  Uaivarsity,  Taivaa,  Chiaa 


/ 


The  problea  can  be  stated  as  find  the  values  of  the  vectors  e  aad  a  so  that  the 
least  square  expression 

J  -  I 

j«l  s»l  J  ■  ■ 

is  uiniaized  subject  to  the  constraints  of  equations  (1)  and  (4) .  This  is  a 
■ultipoint  boundary  value  probleu  with  uiniaization.  It  can  be  solved  by  the  use  of 
quasilinearization  .  Equations  (1)  and  (4)  can  be  coabined  to  obtain 

||  ■  9<y.t)  (6) 

where  y  and  q  are  M  -t  L  diaensional  vectors.  Equation  16)  can  be  linearized  by 
the  use  of  Taylor  Series  with  second  and  higher  order  terns  oaitted.  The  resulting 
vector  equation  is 

— jj-  “  g(y|5.  t)  +  ■  y^)  n) 

where  yj^  is  assuaed  known  and  is  obtained  froa  the  previous  iteration  and  ’^k+l 
the  unknown  function.  The  expression  JCyj^)  is  the  Jacobian  aatrix.  Because  of  the 

fast  convergence  rate,  Equation  (1)  with  unknown  initial  conditions  can  be  solved 
quickly  by  the  use  of  the  superposition  principle.  In  general,  less  than  ten 
iterations  are  needed  to  obtain  a  very  high  accuracy. 


TEX  IRTiriCZAL  KISKXr  STSTXK 


Consider  the  following  siaple  aodel  of  the  artificial  kidney  systea  [4,  10]. 

V,  -rr  -  0  -  K(Cj  -  Cj)  (8) 


'1  dt 
dCj 

'’2  dt  ■  ""1 

where  6  ■  urea  (or  creatinine)  production  rate 


K(C, 


<=2’  -  'k 


V2 


(9) 


k  ■  nass  transfer  paraaeter 

C|^  ■  clearance  rate  of  patient  kidney 

■  dialyzer  clearance 

■  urea  concentration  in  intracellular  cell 
Cj  ■  urea  concentration  in  extracellular  cell 

■  voluae  of  intracellular  cell 


volume  ot  extracellular  call 
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Za  actual  azpariaaatal  aitnatioas,  tha  eoaataata  or  paraaataca  caaaot  ba 
■aasarad,  oaly  caa  ba  aaaaurad  at  tba  Tarieua  raloai  of  t.  Our  problaa  la  to 

aatiaata  k  aad  C^(0)  for  Cduatiooa  (S)  aad  (9)  froa  tha  aapariaaatal  data 

Cj‘“**’(t,)  -  Cj,.  a  -  1,2 . a  (10) 

Rotica  that  tha  iaitial  eoaditioa  of  C^CtaO)  caa  ba  aaaaurad,  hut  C^(t«0)  auat  ba 

aatlaatad.  Thua,  aa  aquatloa  Ilka  iRuatioa  (4)  caa  ha  aatahliabad  for  tha  paraaatar 
k. 


Thia  problaa  ia  aolaad  by  quaailioaariaatiea  with  tha  follewiao  aapariaaatal 
data  [4] 

C^(aap)(j.^,l)  ,  2,070^ 
g^<aap) j  gjg 


-  1.674 

aad  tha  waluaa  of 

6  ■  0,031,  Cg  -  3.6,  Cg  -  0,  at  -  0.01, 

C2(t>0)  >  2.538,  ■  3 

four  diffaraat  axpariaaata  wara  carriad  out  with  four  diffaraat  aata  of  iaitial 
approxiaatioaa.  Tha  coavargaaea  ratas  ara  auaaarixad  ia  Tahla  1.  Notiea  that  fita 
digits  accuracy  ara  ohtalaad  ia  6  to  10  itaratioas.  Tha  kuaga-Kutta  iatugratioa 
tachaigua  was  usad. 


chucosi;  >110  mmm  khibtics  Mopniiic 


Coasidar  tha  followiag  siapla  oaa  CMpartaaat  aodal  of  glucosa  aad  iasulia  in 


plasaa 

[5,  6] 

8  -  *  II"  *  ^3«  *  ^2 

(11) 

II  •  -  IgO  -  IjH  ♦  Ij 

(12) 

wbara 

C  ■  plasaa  glucosa  coacaatratioa 

B  ■  plasaa  IKI  coacaatratioa 

Z^  ■  paraaatars  or  eonstaats. 

Tha  problaa  is  to  astiaata  I^,  Ij,  X^,  ig,  l(t»0)  aad  0(t*0)  froa  asvariaoatal  data 


for  H  aad  0  at  various  waluas  of  t.  kgaia,  aquations  lika  aquation  (4)  caa  ha 
astablishad  for  tba  four  paraaatars. 

Tba  four  paraaatar  waluas  aad  tha  two  iaitial  ooaditioaa  ara  astiaatad  by 
quasiliaaarisatioa.  Tbs  auaarical  waluas  usad  ara 


290 


Ij  ■  -1.S6,  B  6.94,  ■  180  ainutM 

At  -  0.2. 

ThA  •xperiaADtal  data  uaad  art  liatad  In  Tabla  2.  Savaral  diffarant  aata  of  initial 
approxinationa  ara  uaad.  Ona  of  tha  typical  raavlta  ara  liatad  in  Tabla  3.  Tha 
initial  approxinationa  ara  obtainad  by  intagrating  tba  aquationa  with  tha  waluaa  for 
tha  Zaroth  itaration  aa  tha  initial  eonditiona.  Tha  Runga-Kutta  taehniqva  ia  again 
uaad.  Notica  that  avan  with  tha  wary  axtrana  initially  aaaunad  initial  eonditiona 
of  zaro,  only  nina  itarationa  ara  naadad  to  obtain  a  fiva  digita  accuracy. 

CmiOVASCULRR  utoicrtor  dilotiow  mooeliiig 


Conaidar  the  following  four  call  eardiovaacular  indicator  dilution  nodal  [7,  8]. 
*<=1 

-dt  •  “l^l  * 
dC, 


-df-®i«:i- V 

4C, 


■df  •  »1<S 

*<=4 

-df 


<=4> 


(13) 


whara  B, 


r/v, 


*2  “  ”*** 


r  B  Tolunatric  flow  rate 
■  racycla  volunatric  flow  rate 

V  ■  voluna  of  tha  wall-nixad  calla 
Tha  boundary  conditions  for  Equation  (13)  ara 


Cj(t*0) 

CjCt-O) 


B 


3' 


Cj(t-O)  -  0 
C^{t-0)  -  0 


(14) 


whara  M  ia  tha  naaa  of  tha  injaction  and  tha  C^'a  ara  tha  concantrationa  of  tha 
eorraaponding  calla. 

In  actual  azparinanta,  only  tha  Ca  can  ba  naaaurad,  tha  paranatara  Bj^  and  Bj 


cannot  ba  naaaurad  diractly  and  nuat  ba  aatinatad  indiraetly  fron  axparinantal  data. 
Tha  valuaa  of  B^,  Bj  and  B^  ara  aatinatad  by  quaailinaarization  with  tha 

nuaarical  data  liatad  in  Tabla  4.  Tha  Runga-Kutta  nunarical  intagration  foraula 
with  At  ■  0.2  ia  uaad.  Tarioua  diffarant  initial  approxinationa  for  B^,  Bj  and  Bj 

warn  uaad.  Tha  eonrarganea  rata  ia  again  wary  faat.  Thraa  typical  conwarganca 
rnawlta  ara  liatad  in  Tabla  5  for  thraa  diffarant  aata  of  initial  approxanationa. 


1 


: '  i  ♦  *  4  f i  ;  MVJ  f  ^  J  ^  #  Vi't 


Cottsidtr  th«  following  phanueekinotie  nodal  unad  to  pradiet  tka  datailad 
diatribntion  and  axeratioa  of  aathotraxata  in  naaaalian  apaciaa  ovar  a  wida  ranga  of 
doaaa  [9].  Tba  aatarial  balanca  agnations  raprasanting  tka  various  anatonical 
conpartaants  ara 


Plasna: 

% 

"dt  ■  ®L  Rj^  *  ®R  Rjj 

*  Cm  R„  - 

‘Cl  ^  h  *  Cg)  S 

(15) 

Kuscla: 

«=p  -  1^’ 

(16) 

Kidnay: 

’^R 

-df  -  fiR  ‘S  -  s' 

(17) 

Livar: 

"l 

"4?  “  <«L  -  «G» 

-  J*)  -  f 

R-  R, 

Q  L 

(18) 

Gut  Tissua: 

\  -df  -  «G  ‘S  - 

Cf. 

—)  ♦  1/4 

“g 

*  >'c 

^  ‘r-rr  * 

i-i  h  ^ 

(19) 

Gut  Lunan: 

-it 

(20) 

'.a  Ifi.,  .. 

4  dt  S  *r 

’'gl  Ci- 

’'e 

.  c|  •  'l’ 

(21) 

^  'ii.k  V 

4  dt  *F  Sl 

(Ci.i  - 

'  -  "*%  .  c  •  ‘  'i' 

V  1 

(22) 

i  -  2,3,4 

wkara  the  value  of  r  in  Equation  (18)  can  ka  raprasantad  by 
K,  (C./R.) 

-  vTcjTiJ  <2 

wkick  is  tka  sseration  rata  of  natkotraxata  out  of  tka  livar  calls  into  tba  bila 
duets.  Using  tba  tkraa  eoapartnants  nodal,  wa  bava 

*dt  “  '  ■  'l 


■^1  ■  '2 


T  -  rj  -  rj  (26) 

wkara  C  is  tka  drug  eoncantration  in  tka  various  anatonical  eenpartnants,  r  is  tka 
drug  transport  rata  in  tba  bila,  V  is  the  voluna  of  tka  various  eonpartnants,  b  is 
tba  rata  constant  for  nonsaturabla  gut  absorption,  Q  is  tka  plasna  flow  rata,  R  is 


.4 

-■rL  ..  . 
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th«  tiaitt*  pluM  tquilibrlua  ratio  for  linoar  binding  and  Kj^  is  kidnay  elaaranea 
and  is  agval  to  1.1  ni/nin  for  rat.  Tho  otbar  nunarical  raluas  osad  for  rat  ara: 


V_  ■  9  ni 
P 

-  100  a/ 

-  1.9  n/ 

•  8.3  ni 

Vg  -  11  nf 
Vgi.  "  11 
Q|l  ■  3  njf/nin 


Qj^  ■  S  >//nin 
>  6.S  ni/nin 
Og  -  5.3  B//nln 

-  3.0 

«L  "  3-® 


Tha  body  waigbt  for  rat  is  200  g.  Notica  that  thrae  conpartnents  wars  assunad  lor 
bila  sacration  and  4  conpartnants  ware  assunad  for  gut  lunan.  Sons  of  tha 
paranatars  such  as  Rg,  kg  and  Kg  are  not  neasurabla.  These  paranatars  for 


nathotrasata  in  rat  will  ha  astinated  by  quasilinaarization  using  ezparinantal  data 
obtained  by  Bisehoff  at  al.  [9].  These  ezpariaental  data  as  a  function  of  tine  for 
the  drug  concentrations  in  tha  various  conpartnents  are  listed  in  Table  6  and  are 
obtained  Iron  the  figures  of  reference  [$]. 

It  should  be  anphasizad  that  tha  paranatars  k.  and  K.  cannot  be  astinated 

9  V  V 

easily.  This  is  because  that  the  systens  of  differential  aquations  cannot  be  solved 

in  closed  fora,  thus,  quasilinaarization  foms  an  ideal  and  powerful  approach. 

In  addition  to  tha  13  differential  equations  represented  by  Equations  (15)  - 

(26) ,  3  additional  differential  equations  in  the  forn  of  Equation  (4)  can  be 

fomulated  for  the  3  unknown  paraneters.  Thus,  there  are  a  total  of  16  differential 

equations.  The  initial  conditions  for  the  13  differential  equations  are  all  equal 

to  zero  except  C.(t)  which  is 
P 

C  (t)  ■  1200/9  (27) 

P 

The  16  different  equations  can  be  linearized  by  using  Equation  (7).  The  unknown 
paraneters  can  then  be  obtained  by  using  Equation  (5)  and  superpositoin  principle. 
The  hoaogeneous  and  particular  solutions  cau  be  obtained  by  nunerically  integrating 
tbe  linearized  equations.  In  the  present  work,  the  nodified  kdaa-Houlton 
integration  sehene  is  used  with  step  size  as 

at  ■  0.01  ninute  for  0  a  t  s  30 
at  -  0.1  ninute  for  30  a  t  a  240. 

The  convergence  rates  for  the  three  paraneters  are  listed  in  Table  7.  Potiee  the 
fast  convergence  rates.  Only  5  iterations  are  needed  to  obtain  4  digits  accuracy. 
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Sinct  tii«  rtculta  of  tho  proTioui  itorotloa  for  oil  t  oust  bo  otorod  in  tho 
eoaputor,  tho  otoroao  roquiroMot  can  bo  ^vito  lorpo.  For  ouaplo,  tho 
phoraocokinotie  aodol  aooda  (30/0.01  *  210/0.1  1)  Id  >  01616  atbrago  apaeoa.  In 

ordar  to  rodneo  thia  atorago  roguiroaont.  «o  can  atom  only  tho  initial  eonditiona 
of  tho  proolotta  itoration.  Tho  eoaploto  profilo  for  all  t  of  tho  prooioua  itoration 
can  bo  obtainod  by  intograting  tho  ognationa  ahon  wo  ealeulato  tho  enrront 
itorationa.  Tho  atorago  rogviroaoata  can  thoa  ho  rodveod  troaondonaly.  For  tho 
pharaaeokinetie  probloa,  tho  atorago  roguiroaont  ia  radueod  froa  81616  to  16. 
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Tibl*  1  Convarganee  Rataa  of  tha  Artificial  Kidnap  Model 


Iteration  | 

Cj(0)  1 

i  *  i 

1 

K 

1 

1  '  1 

Cj(0) 

K 

0 

1  2.S38 

5. 

2.538 

112. 

1  2.538 

1  19.2  1 

2.538 

1  25. 

1 

1  2.9S13  1 

6.1718 

2.7879 

1  5.2057 

1  3.1695 

1-35.947  1 

2.4352 

1  18.639 

2 

1  2.767$  1 

7.5204 

2.8314 

!  7.4735 

1  2.9149 

1-  4.79061 

3.1274 

1-34.37 

3 

1  2.7997  1 

7.5318 

2.8023 

1  7.4970 

1  2.9895 

1  6.96271 

2.7892 

1-  7.5045 

4 

1  2.8000  1 

7.5279 

2.7994 

t  7.5351 

1  2.7776 

1  7.5923  1 

2.6165 

1  5.6438 

S 

1  2.7999  1 

7.5288  1 

2.8000 

1  7.5272 

1  2.7991 

t  7.5369  1 

2.8398 

1  7.8420 

6 

1  2.7999  1 

7.5286  1 

2.7999 

t  7.5289 

1  2.8000 

1  7.5270  1 

2.8016 

1  7.5273 

7 

1  2.7999  1 

7.5286  1 

2.7999 

1  7.5285 

1  2.7999 

1  7.5290  1 

2.7999 

1  7.5292 

S 

1  1 

1 

2.7999 

I  7.5286 

1  2.7999 

t  7.5285  1 

2.7999 

(  7.5285 

9 

1  1 

1 

2.7999 

1  7.5286 

1  2.7999 

1  7.5286  1 

2.7999 

1  7.5286 

—12 _ ! 

1 _ 1 

_ 1 

J _ 

1  i-^999 

1 

him 

1  7.5286 

Table  2  Kzpariaantal  Data  for  Glueoaa  and  Inaulin  Xinetiea  Modal 


j(SSp) 

C“*>'»(ts) 

0 

177 

581 

30 

155 

182 

60 

40 

95 

90 

26 

87 

120 

20 

97 

150 

24 

106 

180 

28 

no 

Table  3  CooTacgenca  Katea  of  Clueoaa  and  Insulin  Kinetics  Model 


Iteration 

^1 

^2 

^4 

^6 

B(0) 

G(0) 

0 

0. 

0. 

0. 

0. 

177. 

581. 

1 

0.051076 

0.025872 

0.048153 

0.22224 

181.31 

576.58 

2 

0.038405 

0.017182 

0.020605 

0.052089 

177.16 

580.37 

3 

0.045445 

0.021543 

0.028009 

0.043957 

177.56 

580.06 

4 

0.046151 

0.022149 

0.028790 

0.043174 

177.27 

580.35 

5 

0.046411 

0.022281 

0.028581 

0.043500 

177.24 

580.38 

6 

0.046408 

0.022286 

0.028565 

0.043510 

177.23 

580.39 

7 

0.046423 

0.022293 

0.028555 

0.043523 

177.23 

580.39 

8 

0.046421 

0.022292 

0.02855$ 

0.043523 

9 

0.046422 

0.022293 

0.028555 

0.043524 

_ 10__ 

0.022293 

0.028555 

0.043524 
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Tablt  4  Kzp«riB«ntal  Data  for  CardloTaaealar  Hodal 


Cj(t.) 

C,(t.) 

w 

<=4<S» 

0.0 

0.9997 

0.0 

0.0 

0.0 

2.0 

0.2289 

0.3314 

0.2609 

0.1387 

4.0 

0.1327 

0.1887 

0.2391 

0.2366 

6.0 

0.1141 

0.1347 

0.1682 

0.2009 

8.0 

0.0909 

0.1066 

0.1269 

0.1528 

10.0 

0.0702 

0.0834 

0.0988 

0.1175 

12.0 

0.0543 

0.0646 

0.0768 

0.0912 

14.0 

0.0421 

0.0501 

0.0595 

•  0.0707 

16.0 

0.0327 

0.0388 

0.0462 

0.0549 

18.0 

0.0253 

0.0301 

0.0358 

0.0425 

20.0 

0.0196 

Table  5  Convergence  Rata  of 

Cardiovascular  Model 

Iter¬ 

ation 

81 

82 

63 

1  81 

82 

83 

81 

82 

83 

0 

0,1 

0.01 

0.1 

0.6 

0.2 

0.8 

2. 

1.5 

3 

1 

0.4379 

0.0725 

0.4969 

0.7663 

0.3755 

0.9903 

1,7167 

1.2619 

1.0049 

2 

0.4896 

0.1772 

0.8993 

0.7966 

0.3970 

0.9992 

0.6846 

0.2886 

0.9983 

3 

0.6522 

0.2679 

0.9658 

0.8013 

0.4015 

0.9996 

0.8014 

0.4021 

0.9992 

4 

0.7616 

0.3635 

0.9952 

0.8017 

0.4018 

0.9997 

0.8017 

0.4018 

0.9997 

5 

0.7974 

0,3979 

0.9993 

0.8017 

0.4018 

0.9997 

0.8017 

0.4018 

0.9997 

6 

0.8014 

0.4015 

0.9997 

7 

0.8017 

0.4018 

0.9997 

8 

0.8017 

0.4018 

0.9997 

Table  6  Exparimantal  Data  for  Fbaraacokinatica  Modeling 


*. 

(Sin) 

Cp(t,) 

15 

7,7 

1.5 

20. 

20.9 

23.98 

30 

4.0 

0.75  • 

10.8 

11.5 

47.00 

60 

1.5 

0.25 

4.0 

4.97 

59.00 

90 

1.14 

0.16 

2.8 

3.60 

45.50 

120 

0.80 

0.13 

2.2 

2.80 

36.00 

180 

0.45 

0.072 

1.1 

1.45 

18.25 

_24fi _ 

_ 0.27 

_ 0.043 

_ JL47 _ 

_ _ 

_ tLlfi 

Table  7  ConTergeace  Rates  of  Pbarsacokiaetics  Model 


Itaration 

*0 

‘o 

■'o 

0 

1. 

20. 

200. 

1 

1.108 

22.64 

237.2 

2 

1.112 

21.61 

224.6  . 

3 

1.112 

21.97 

229.3 

4 

1.112 

21.85 

227.7 

5 

1.112 

21.89 

228.3 

6 

1.1» 

_ jua _ 

■3«J _ 

A  THREE-MIRROR  PROBLEM  ON  DYNAMIC  PROGRAMMING 
Sellchi  Iwanoto 

Department  of  ftsonomlc  Engineering 
Faculty  of  Economics 

Kyushu  University  27,  Fukuoka  812,  Japan 

1 .  INTRODUCTION 

The  essence  of  dynamic  programming  states  that  a  simultaneous 
optimization  of  real-valued  two-varl2U3le  functions  Is  assured  by  the 
two-stage  optimization  under  both  separability  and  monotonlclty  [l5, 
16^.  We  call  these  two  properties  the  recuslveness  with  monotoni¬ 
city  -  dynamic  progreunmlng  structure  -  [8,  1l]  .  This  struc¬ 

ture  yields  what  we  call  dynamic  programmable  function  [l l] . 

In  this  paper  we  focus  our  attention  on  both  dyneunlc  program¬ 
ming  structure  and  guasllllearlzation  for  a  class  of  objective  fun¬ 
ctions.  Given  a  differentiable  strictly  Increasing  convex  function 

f  ;  R^ . ■■■■»  R^ ,  we  approximate  f(x)  by  its  linear  approximation  f(x;h) 

111  1 
R  xR  - »  R  ,  which  is  strictly  Increasing  in  h  for  x  e  R  .  Thus , 

f(x)  is  a  guaslllnearizatlon  of  f(x;h).  The  N-times  composition  of 

N  1 

f(Xjj;")  generates  a  dynamic  programmable  function  F(x;h)  :  R  xR  — 

-»  r\  Similarly,  Inverse  function  f”^ <y) ,  reverse  function  f_^ (x;k) 

which  is  the  Inverse  function  of  f(x;h)  with  respect  to  h  for  fixed 

* 

X,  and  conjugate  function  f  (y)  also  generate  dynamic  programmable 

functions  f”^  (y;k) ,  F_^  (x;k)  ,  and  P*(y;h)  :  R**xr’  - r\  respeot- 

tively.  Thus,  the  function  f  yields  four  -  main.  Inverse,  reverse, 

emd  conjugate  -  optimization  problems  on  R  .  These  problems  are 

solved  through  dynamic  programming  approach.  Some  relations  between 
them  are  established.  Finally  we  Illustrate  two  Interesting  ex«unples 
from  Bellman  [l]  . 
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s.t.  — “  <x  <<”  VSnSN 

n 

where  h  is  a  real  constant.  We  remark  that  the  N-times  iteration  of 

f  (x;h)  =  e^(1  -  X  +  h) 
yields  the  objective  function 


f  (x^  ;f  (Xj; . . .  ?f  (Xjj.’h)  . . . )  i 


(See  also  [11,  p.278:  12,  p.285]). 

Second  we  consider  the  following  maximization  problem; 

Max  (1-2x^^)exp(x^^)  +  2x^  (l-2x2^)exp(x^^+X2^)  + 

* (1-2x2^)exp(x^^+X2^*Xj^)  ♦  8x^X2X2 ( 1-2x2^) 
2  2  2 

xexp(x^  ♦X2  ♦Xj  )h 
s.t.  x^£0,  X22O,  x^iO 
where  h  2  0.  The  three-times  iteration  of 
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f(x;h)  =■  (1  -  2x^  +  2xh)exp(x^) 

generates 

f  (x^  ;f  (Xj.'f  (x^.'h) ) ) 

=  (1-2x^^)exp(x^^)  +  2x^ exp (x^^)  {^(1-2x2^) exp (X2^)  +  2x2exp(x2^)x 
[(1-2X2^)  exp  (Xj^)  +  2x2exp(x2^)h]3  . 

These  two  functions  are  called  reouraive  funotiona  on  B^freap. 

R^)  with  atriat  inareaaingneaa  ((jlO,  1 1 J  )  .  A  function  P  :  — 

1  N 

-»  R  Is  called  dynamic  programmable  function  on  R  If  it  is  expressed 

as  follows 

P(X^  fX^i • «  •  f X^  7  h ) 

»  f^  (x^;f2(x,,X2;...;fj,(x^,X2,...,Xj,;h)...)) 

where  f _ :  r"xR^  - »  R^  and  f_ (x, ,x-, . . . ,x_;  R^  - *  R^  is  non- 

n  n  I  4  n 

decreasing  for  1  S  n  £  N,  (x^  ,X2 » •  • .  ^x^^)  e  r”.  Therefore,  any  re¬ 

cursive  function  with  strict  increasingness  is  a  dynamic  programmable 

function.  In  the  following  we  are  mainly  concerned  with  a  class  of 

N 

recusive  functions  on  X(cR  )  with  strict  increasingness. 

3.  MAIN  RESULT 

First,  we  prepare  the  following  fundeunental  lemma.  Let  X  and 
Y  be  two  nonempty  sets.  For  each  x  e  X  let  Y(x)  be  a  nonempty 

subset  of  y.  That  is,  Y(-)  ;  X  — ►  2^  is  a  point-to-set-valued 

Y 

mapping,  where  2  denotes  the  set  of  all  nonempty  subsets  of  Y. 

Let 

Gj.(Y)  «  { (x,y)  I  y  c  y(x) ,  x  c  X}  c  Xxy 
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be  the  graph  of  the  mapping  Y ( • ) .  In  the  following  It  will  be  clear 
from  the  context  whether  a  notation  Y  Is  considered  the  set  or  the 
mapping . 

LEMMA  1  (Maxlmax  Theorem  [ll;  p.268])  Let  f  ;  be 

a  function  such  that  f(x;*)  :  R^  — »  R^  Is  nondecreasing  for  x  p 

X.  Let  g  :  G  (Y)  - »  R^  be  a  function.  If  Max  f(x;  Max  g(x,y)) 

^  xex  yeY(x) 

exists,  then  Max  f(x;  g(x,y))  exists  and  both  are  equal: 

(x,y)eGj.(Y) 

Max  f(x;  Max  g(x,y))  >=  Max  f(x;  g(x,y)). 

xeX  yeY(x)  (x,y)eG^(Y) 

REMARK  This  equality  remains  valid  even  If  the  operator  Max 
Is  replaced  by  the  operator  min  under  the  same  condition  as  stated 
above.  Furthermore,  as  a  special  case  we  have 

Max  f(x;  Max  g(y))  “  M^uc  f(x;  g(y)). 

•cD<x<®  — oo<y<ao 

In  general  we  have  for 
any  differentiable  convex 
function  f  ;  R^  - *  R^ 

f (h)  »  Max  f(x;h)  (1) 

-»<x<« 

where 

f(x;h)  ■  F(x)  ♦  f'(x)h 

(2) 

F(x)  -  f (X)  -  xf' (X) . 

Thus,  f(x;h)  Is  the  linear  approximation  of  f(x)  at  hi 


-“><x,y<“> 


f(x;h)  -  £(x»  ♦  (h-x)f'(x).  (3) 

The  expreaalon  (1)  la  called  a  quaeilintaTitation  of  f(x)  (^1;  p.135 
;  13;  14]). 

Furthermore,  from  Lemma  1,  we  obtain  under  £'(x)  2  0,  -«  <  x 

<  «0 

f(£(h))  ■  Max  £(x.;  Max  £{x,;  h) ) 

-•“<X^«»  '  -aXXjO®  ^ 

(4) 

»  Max  f (X, ;  £(X2;h)) . 

-®<x^ ,X2<“ 

that  la 

f(£(h))  -  Max  fF(x,)  ♦  £'(x,)(Max  [,F(x,)  +  f'(X2)h])j 

-•<x^<®  '  -»<X2<®  ^  ^ 

■  Max  [P(x,)  ♦  £'(x,)F(x,)  ♦  f'(x,)f'(X2)h]. 

-«<x^  »X2«»  I  ^ 

DEFINITION  Let  £  :  - »  R^  be  a  differentiable  increas¬ 

ing  (reap,  strictly  increasing)  convex  function.  Then  we  define  F 

f(X2;  f  (Xjj;h) . . .) ) 

♦  f'(x^)F(X2)  ♦...  ♦  f' (x, )  f' (Xj)  . .  .f '  (Xjj_^ ) 
xF(Xjj)  ♦  f'(x,)f'(x2)...f'(Xjj)h 

where  f(x;h)  and  F(x)  are  defined  in  (2),  and  x  ■  (x^,X2,..., 

N  1  *1 

Xjj) .  The  function  F  :  R  xr'  »  r'  is  the  rsoursivs  function  with 

inarcaeingncta  (rctp.  strict  inorcasingncss)  generated  by  f  or  simply 


:  R"xr'  - f  R'  by 

F(x;h)  ■  f(x^; 
■  F(x^) 
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dynamio  programmable  function  generated  by  /. 

Xn  the  following,  It  will  be  clear  from  the  context  a  function 
f  (reap.  F)  Is  considered  f (x)  or  f(x;h)  (resp.  F(x)  or 
F(x;h)) . 

REMARK  The  equalities  (1)  and  (4>  (qr  (5))  remain  valid 
If  we  replace  'Max'  and  'convex'  with  'min'  and  'concave',  respec¬ 
tively.  Similarly,  a  differentiable  Increasing  (resp.  strictly 

Increasing)  aonaave  function  g  :  R^  - *  R^  generates  the  reaursive 

N  1  1 

function  G  :  R  xR  - >R  with  increasingnees  (resp.  strict 

increasingneas) ,  which  is  also  called  dynamic  programmable  function 
generated  by  g; 

G(y;k)  «  g(y^;  <3(^2'  *•*'  9(y}}?i«>  •  •  • ) ) 

■  G(yi)  *  <3' ^y■^>G^y2^  *  - 9'<yN«i) 

''G(yj,)  +  g'(y^)g'(y2)-*.g'<yi,)i^  (7) 

where 


y  “  ^y^'y2 . 

gCyjR)  “  g(y)  +  (H  -  y)g'(y) 
*  G(y)  +  g'(y)k, 

G(y)  »  g(y)  -  yg' (y) . 


(8) 


Therefore  we  have  the  following  main  result: 

THEOREM  1.  (1)  Let  ft  »  R^  be  a  differentiable  Increas¬ 

ing  convex  function.  Then  for  h  e  R^ 
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ctlon  generated  by  £~^  and  f'^^Cy)  is  the  n-tlme  composition  of 


(ii)  Let  g  ;  R' 


r'  be  an  onto  differentiable  strictly 


increasing  concave  function.  Then  for  h  e  R 


g~^’(h)  ■  MaXj,  g“^  (x;h)  ^,2) 

X£R 

*  «N4-1  *  *  -1  * 

and  »  g  (h)  ,  X2  =  g  '^(h),  Xjj_^  =  g  (h)  ,  x^  »  h  att¬ 

ains  the  maximum,  where  G~^ (x;h)  is  the  dyneunic  programmable  fun¬ 
ction  generated  by  g  ^ . 


Here  we  remark  that 


F"''(yjk)  »  F"’(y,)  +  f"’'(yi)F~'' (yj)  +  ...  ♦  f""*  *  (y^ )  f”'* '  (yj) 


x...f-’'(yj,_^)F‘^yj,)  ♦  f'’'(yi)f‘’'(y2)...f'’*(yjj>>^ 


where 


f"''  (y)  -  f*’  (y)  -  yf~’'(y) 


and  f”^'  is  the  derivative  of  the  inverse  function  f~\  Similarly, 


G~  (xjh)  is  defined  and  omitted. 


Second  we  consider  the  reversion  of  the  linear  approximation 


£(x;h)  of  f(x) 


not  the  reversion  of  f (x)  itself 


follows.  For  any  onto  differentiable  strictly  increasing  convex 
function  f  s  R^  — ♦  ,  its  linear  approximation  £  ;  R^  xR^  — •  R^ 


L. 
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defined  by  (2)  or  (3)  Is  continuous  strictly  increasing  and  linear 
in  h  for  X  e  .  Therefore,  f(x;*)  :  — »  R^  is  invertible  for 

X  e  R^ .  Its  inverse  fanction  f_^(x;.)  :  R^  — ►  R^  becomes 


f-1  (X) 


(15) 


where 


F.i(x)  =  X  (16) 


We  call  f_^  *  f_^ (x;k) 

the  reverse  funotion  of 
f  >  f(x;h).  As  we  noted 
in  ( 1 ) ,  we  have 


f  (h)  »  Max  f  (x;h) 
-®<x<* 


■  Max  [Fix)  +  f'(x)h3 
.ce<x<<>> 


(17) 


•  Max  [f(x)  +  (h  -  x)f'(x)] 

“*<X<* 


and  X  >  h  attains  the  maximum.  This  fact  is  equivalently  trans¬ 
formed  to 


f”^ (k)  ■  min  f_. (x;k) 


min 
-•<x< 


(18) 


/ 
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■  min 


r .  .  k  -  fix) 

1*  *  -rntn- 


and  X  =  f”^ (k)  attains  the  minlmiun  (see  Fig. 2).  This  fact  ref¬ 
lects  also  the  main  Idear  of  Newton  method  from  a  viewpoint  of  opti¬ 
mization.  Therefore,  we  have  the  following  reversed  form  of  (9) : 

THEOREM  2.  (i)  Let  f  :  - *  s}  be  an  onto  differentiable 

strictly  increasing  convex  function.  Then  for  k  e  R^ 


f~^(k)  =  min„  F  , (x;k) 
xeR^  ■’ 


(19) 


and  x^  =  f’*’(k),  Xj  =  f"*^"'' (k) ,  ...,  Xjj_,  =  f"^(k),  Xj,  =  f"’ (k) 
attains  the  mindLmum,  where  P_^  :  R^xR^  — ►  R^  is  the  N-times  com¬ 
position  of  f_^(x;k)s 

F_^  (xjk)  =  f_^(x^;  f__^(x2;  f_^  (Xjj;k) . . . ) ) .  (20) 

(ii)  Let  g  :  R^  — * R^  be  an  onto  differentiable  strictly 
Increasing  concave  function.  Then  for  h  e  R^ 

g"*’(h)  =  Max„  G  ,  (y;h)  (21) 

yER”  * ' 

and  y*  ■  g“^(h) ,  th)  •  •••»  y^-i  “  g"^<h)i  “  g”^  (h) 

attains  the  maximum,  where  G_^  :  R^xR^  - ►  R^  is  the  N-times  com¬ 

position  of  g_^ (y;:  )  : 

G"^  (y;h)  »  g_^(y,;  9_^(y2»’  •”’>  g_,  (Xjjjh)  . . . ) )  .  (22) 


Here  we  remark  that 
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where  P_^ (x)  is  defined  in  (16).  Similarly,  G_^ (y;h)  is  de¬ 
fined  from  (y^j)  ,  g'Cy,,)  and  h.  We  call  P_^  (x;k)  ,  G_^  (y;h) 

the  dynamia  programmable  funation  generated  by  reverse  function 
f_^(x:k),  g^^(y;h)),  respectively. 

We  have  the  following  relation  between  P  ^ (y;k)  and  F_^ 

(x;h) : 

THEOREM  3.  (i)  Let  f  :  ——*■  R^  be  an  onto  differentiable 

strictly  increasing  convex  function.  Then  we  have  by  the  monotone 
transformation  y  »  f (x) 

f"''  (y;k)  «  f_^  (x;k).  (24) 

Purthemore,  the  monotone  transfomation  y  =f(x)1SnSN  yields 

P"''  (y;k)  -  F_,  (x;k).  (25) 

(ii)  Let  g  :  R^  — *  R^  be  an  onto  differentiable  strictly 
increasing  concave  function.  Then  we  have  by  the  monotone  trans¬ 
formation  X  ■  g(y) 

g"^ (x;h)  -g_^(y;h).  (26) 

Furthermore,  the  monotone  transformation  x^  ■  g(y..)  1  S  n  S  N 

n  n 

yields 
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G"'*(x;h)  »  G_^(y;h).  (27) 

Pr'oof.  It  is  straightforward. 

Finally  we  consider  conjugations  *  and  ^ .  For  any  convex 

11  *  1 
function  f  !  R  - ►  R  ,  we  define  its  aonjugate  function  f  ;  R 

— .R^  ■ 

f*(y)  =  Sup  [xy  -  f (x)]. 

On  the  other  hand,  for  any  concave  function  g  ;  R^ 
denote  its  aanjugate  function  »  R^  by 

g(x)  =  inf  (yx  -  g(y)]  .  (29) 

— flp<y<» 

If  both  operations  *  and  are  well  defined,  they  are  dual 
in  the  following  sense: 

(■^)  (y)  =  -f*(-y)  y  e  r\ 

LEMMA  2.  Let  f  :  R^ - -  R^  be  a  twice  differentiable  strictly 

Increasing  strictly  convex  function.  Then  we  have  for  f'(-'»)  <  y 
<  f'(“) 

(i)  f*(y)  -  xy  -  f(x) 

(ii)  f*'(y)  ■  X  and  in  particular  f*'(y)  >  0  for  f ' (0) <y<f ' (») 

and 

(iii)  f*"(y)  -  jTT^  >  0 

where  x  satisfies  uniquely  f'ilx)  ■  y.  Therefore,  f*  :  (f'(0). 


/ 
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£'(«>))  — *  l8  Strictly  Increasing  strictly  convex.  Tt^us  we 

* 

have  the  following  result  for  f  : 

THEOREM  4.  Let  f  :  R^  — *  R^  be  a  twice  dlf ferentlaUsle 
strictly  Increasing  strictly  convex  function.  Then  we  have  for 
f'(0)  <  f*”(h)  <  £'(-)  0  1  n  S  N-1 


f*^h) 


Mm 

(0)<y„^f'(») 


IlnlM 


F  (y;h) 


(30) 


and  y 


y; 


f*<«-2)(h) 


attains  the  maximum,  where  F  (y;h) 


function  generated  by  f 


and  f 


'  ^N-l  “ 

Is  the  dynamic  programmable 
Is  the  n-tlme  composition  of 


Similarly,  for  concave  function  g,  we  have  the  following; 

LEMMA  3.  Let  g  :  R^  — *  R^  be  a  twice  differentiable  st¬ 
rictly  Increasing  strictly  concave  function.  Then  we  have  for 
g'(«)  <  X  <  g'(-“>) 

(1)  g(x)  »  yx  -  g(y) 

(11)  g'(x)  ■  y  and  in  particular  g'(x)  >  0  for  g' (“) <x<g' (0) 

and 

(ill)  |"(x)  -  <  0 

where  y  satisfies  uniquely  g'(y)  •  x.  Therefore,  g  :  (g' (<»)  ,g' (0) ) 
— >  R^  Is  strictly  Increasing  strictly  concave. 

THEOREM  5.  Let  g  :  R^  - » 


R^  be  a  twice  differentiable 


strictly  increasing  strictly  concave  function.  Then  we  have  for 
g'(=0)  <  g"(k)  <  g'(0)  0  S  n  S  N-1 

g*’(k)  a  min  3(x;k)  (31) 

g' (*) <x^<g' (0)  1£nlN 

and  x^  -  g**~^  (k)  ,  =  g**~^(k),  =  g(k),  x^j  =  k  attains 

the  minimum,  where  3(x;k)  is  the  dynamic  progreunmable  function 
generated  by  g  and  g*^  is  the  n-tlmes  composition  of  g. 

Here  we  remark  that 


F*(y;h) 


where 


F*(yi)  +  f*'(yi)F*(y2)  *  •••  ♦  f*'(yi)f*'(y2)... 
*f*(yN.i>F*(yjj)  ♦  f*'(y,)f*'(y2)  ...f*'(yj,)h 

(32) 


F*  (y)  »  f*  (y)  -  yf*'  (y) 

-  -  f  (X)  . 


(33) 


Here  x  satisfies  uniquely  f'(x)  «  y.  Similar  expressions  for 
S(x;k)  and  6(x)  are  omitted. 


5.  EXAMPLES 

In  this  section  we  illustrate  explicit  form  of  f(x;h),  F(x;h) 
,  P  ^ (y;k) ,  f_^ (x;k) ,  F_^ (x;k) ,  F*(y{k)  and  others  for  a  given  f(x). 

5.1  f(x)  ■  e*  ;  (-«  ,  •)  — ►  (0  ,  «) 

In  this  case  we  have  the  following  expressions.  First  we 


have  from  (2),  (6) 
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f(x;h)  » 

F(x;h)  ■ 

Second . 


(1  -  X  +  h)e  -«  <  x.h  < 


X,  X. +  . .  .+x„  , 

e  ^  (1  -  x^)  +  e  ’  "^(1  -  Xj)  +  ...  +  e  ’ 


!(1  -  Xjj)  +  e 


x,  +  ...  +Xj, 


xh  -®  <  x_,h  <  «>. 
n 


for  Inversion,  we  have  from  (13), (14) 


g(y)  p  f”^ (y)  *  logy  :  (0,  ®)  - ►  (-*,  ®)  (34) 

g(yjk)  ®  f~^  (y.'k)  =  -1  +  logy  ^  0  <  y,k  <  “ 

G(yjk)  »  (y;k)  a  -1  +  logy^  +  <y^)~^<~1  ♦  logyj) 

♦  <-1  +  iogyjj>  ♦  <yi**.yij>"’>‘ 

y^  >  0,  k  >  0 

where  k  0  means  that  k  is  sufficiently  large  that  log...logk 
(N-times  log  operation)  becomes  well  defined.  That  is,  in  this  case, 

..e 

k  >  e®*  ( (N-1) ’s  e) . 

Third,  for  reversion,  we  have  from  (IS) , (16)  ,  (20) 

f_^(x;k)  ■  X  -  1  +  e'^k  -•  <x  <  «,  k  >  0 

-X  -x.-...-x^_- 

F_^ (x;k)  -  x^  -  1  ♦  e  ’ (X2  -  1)  +  ...  +  e  (x^_^-1) 


-•  <x„  «  *,  k  »  0. 
n 


♦  e 
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Moreover,  the  reversion  of  g  >  9(y)  defined  In  (34)  becomes 
g_^  (x;h)  ■  y(1  —  logy)  ♦yh  y  >  0,  <  h  <  « 

G_^(x;h)  -  y^(i  -  logy^)  +  “  iogy2>  *  "•  +  yi*--yN 

x(l  -  logyjj)  +  y^...yjjh  ^  -*  <  h  <  ®. 

Fourth,  for  conjugation,  we  have  from  (28) ,  (29)  ,  (32)  ,  (33) 

f*(y)  ”  (-1  +  logy)y  :  (0,  ®)  — »  Q-1,  *; 

* 

f  '(y)  »  logy  >0  on  (1,  *) 
f*"(y)  «  1/y  >  0 

* 

f  (y;k)  ■  -y  +  kxlogy  y  1 ,  k  >  1 

F*(y;k)  -  -y^  -  yjlogy^  -  ...  -  y^jlogy, . . . logy^,,, 

♦  kxlogy^...  logy  jj  ^  ^  ^ 

g(x)  ■  1  +  logx  :  (0,  “)  - »  (-*,  ») 

g(x;h)  ■  logx  ♦  x”^h  0  <  x,h  <  « 

G(x;h)  *  logx^  ♦  (x^)”^logx2  •*•...  ♦  (x^ . .  .x^_^ )“^ logx^j 

♦  (x^...x„)-’h  *n  >  ^  ° 

where  h  >  0  In  this  case  means  that 

.-l-»e-* 

h  >  e  ’  *  (M's  e). 
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Finally,  for  reversion  of  g(y;k),  we  have 
g_^  lx;k)  »  -xlogx  +  xk  x>0,  -a><)c<« 

(x;k)  ■  -x^logx^  —  x^X2logx2  —  ...  —  x^ . .  .x^^logXjj 
♦  *T--V  *n  "  0'  -»  <  k  <  -. 

5.2  f(x)  -  x^  :  fo,  -)  - >  =>) 

In  this  case  we  have  the  following  result.  First,  we  get 
2 

f(x;h)  =  -x  ♦  2xh  x,h  £  0 

F(x;h)  *  -x^  -  2x^X2  —  ...  —  2**~'x^ . .  .Xjj_^x^ 

♦  2'^x,  ...Xjjh  Xjj  i  0,  h  6  0. 

In  particular  Theorem  1  for  case  M  «  1  Implies 
Max  f  2xh  —  x^j  »  h^  -a,  <  h  <  “. 

This  Is  one  of  the  simplest  guaslllnearlzatlon  [l ;  p.134]. 
Second,  the  Inversion  becomes 

g(y)  a  (y)  ■  ••  (0,  "»)  - >  (0,  “>) 

g(y;k)  ■  f“^ (y;k)  ■  i(  y  ♦  -)  y,k  >  0 

G(y;k)  •  F~^ (y;k) 

yn  >  0.  k  >  0. 
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t 


Therefore,  Corollary  (ii)  for  case  N  »  1 

min  i/x  ♦  — ^  ]  *  /It 
x>0  2/xJ 

(see  also  [l ;  p.134]). 

Third,  for  reversion,  we  have 


f_^ (x;k) 


x,k  >  0 


F_^ (x;k) 


=  Ix^  *  *  •••  *  I 


*  ^n  ^ 


Finally,  the  conjugation  yields 


f*(y)  »  jy^  ••  [O,  «»)  — >  [O,  oo) 


*  12  1 
f  (y;k)  =  -  |y  +  ^yk 


y,k  £ 


F*(y;k)  -  -  -Jy?  -  4^yiy2  - 


7^1  •••yN’' 


4" 

yr 


§(x)  ■  ~  ^  :  (0  ,  <»)  — »■  (-«  ,  0) 


9(x)  “  “  ^ 

4x^ 


x,h  >  0. 


Therefore  we  get 


g(x)  >  min  §(x;h) 

0<X<'» 


reduces 

k  >  0 


ij(xN/Xr--XN_i) 
0,  k  >  0. 

;  0 

_  2 

2N-1^1  •  •  ’^N-nN 

£  0,  k  £  0 


However  if 


N  2  2  ,  then  it  does  not  hold  that 
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g**(h)  “  min  G(x;h)  h  >  0, 

0<x_«» 

n 

because  of  g(h)  <  0. 

References 

1.  R.  Bellman,  Dynamic  Programming,  Princeton  Onlv.  Press,  Prl- 
nston  N. J. ,  1 957. 

2.  R.  Bellm2m  and  R.  Kalaba,  Quaslllnearlzatlon  and  Nonlinear 
Boundary-value  Problems,  American  Elsevier,  N.Y.,  1965 

3.  R.  Bellman  and  Wm.  Karush,  On  a  new  functional  transform  In 
analysis  :  the  maximum  transform.  Bull.  Amer.  Math.  Soc.  67 
(1961),  501-503. 

4.  R.  Bellman  and  Wm.  Karush,  Mathematical  progreunmlng  and  the 
maximum  transform,  J.  SIAM  Appl.  Math.  10(1962),  550-567. 

5.  R.  Bellman  and  Wm.  KARUSB,  On  the  maximum  transform  and  semi¬ 
groups  of  transformations.  Bull.  Amer.  Math.  Soc.  68(1962), 
516-518. 

6.  R.  Bellman  and  Wm.  Karush,  Functional  equations  In  the  theory 
of  dynamic  programming  -  XXI:  an  application  of  the  maximum 
transform,  J.  Math.  Anal.  Appl.  6(1963),  155-157 

7.  R.  Bellman  and  Wm.  Karush,  On  the  maximum  transform,  J.  Math. 
Anal.  Appl.  6(1963),  -67-74. 

8.  N.  FuruJcawa  and  S.  Iweunoto,  Dynamic  programming  on  recursive 
reward  systems.  Bull.  Math.  Statist.  17(1976)  103-126. 

9.  S.  Iwamoto,  Some  operations  on  dynamic  programmings  with  one- 
dlmenslonal  state  space,  J.  Math.  Anal.  Appl.  69(1979),  263- 
282. 

10.  S.  Iwamoto,  Reverse  function,  reverse  rpogram  and  reverse 
theorem  In  mathematical  programming,  J.  Math.  Anal.  Appl.  95 
(1983),  1-19. 

11.  S.  Iwamoto,  Sequential  mlnlmaximlzatlon  under  dynamic  progra¬ 
mming  structure,  J.  Math.  Anal.  Appl.  108(1985),  267-282. 

12.  S.  Iwamoto,  R.J.  Tomlclns  and  C.-L.  Wang,  Some  theorems  on 
reverse  Inequalities,  J.  Math.  Anal.  Appl.  119(1986),  282-299. 

13.  E.  Stanley  Lee,  Dynamic  Programming,  quaslllnearlzatlon  and 
dlmensloraallty  difficulty,  J.  Math.  Anal.  Appl.  27(1968),  303- 


316 


322. 

4.  E.  Stanley  Lee,  Quasillnearlzatlon  and  Invariant  Imbedding, 
Academic  Press,  N.Y.,  1968. 

5.  L.G.  Mitten,  Composition  principle  for  synthesis  of  optimal 
multistage  process.  Operations  Res.  12(1964),  601-619. 

6.  G.L.  Nemhauser,  , Introduction  to  Dynamic  Programming,  John 

Wlely  and  Sons,  1966. 


w 


EXISTENCE  AND  COMPUTATION  OP  SOLUTIONS  FOR  THE  TWO 
DIMENSIONAL  MOMENT  PROBLEM 

Gyorgy  Sonnevend* 

Inst.  fUr  Angewandte  Mathematik, 
UniversltMt  WUrzburg 
0-8700  WUrzburg,  Am  Hubland 

Introduction 


In  this  paper  we  deal  with  some  problems  of  the  theory  of  two 
dimensional .polynomial  moment  problems.  More  precizely  we  give 
necessary  and  sufficient  conditions  for  the  existence  of  a  solution, 

i.e.  of  a  nonnegative  mass  distribution  supported  within  a  fixed,  a 

2 

priori  given  subset  S  of  R  ,  which  has  a  finite  set  of  moments  with 
prescribed  values. VJe  study  the  problem  of  characterizing  all  minimal 
support  solutions,  i.e.  those  solutions  which  have  a  minimal  number  of 
atoms . 

The  connections  between  the  restri«;ted  (or  finite) .  classical , 
polynomial  (onedimensional)  moment  problem  (as  a  special  case  of  the 
moment  problems  of  Nevanllnna-Pick  type)  and  various  other  problems  in 
the  theory  of  orthogonal  polynomials,  rational  Fade  approximation 
(interpolation  of  Stieltjes  functions) , restriction  of  self  adjoint 
operators  to  Krylow-subspaces ,  construction  of  quadrature  formulae, 
minimal  partial  realizations  of  causal  linear  input-output  maps,  are 
well  )<nown.  Similar  applications  for  the  considered  two  dimensional 
generalization  motivate  our  study.  The  method  we  use  for  the  solution 
of  these  problems  is  operator  theoretic  and  is  based  on  solving  an 
"extension  problem"  for  pairs  of  commuting,  self  adjoint  operators .The 
characterization  obtained  for  the  minimal  support  solutions , 1 .e .  for 
the  analogons  of  the  Gaussian  quadrature  formulae  is  different  from 
the  previous  approaches,  which  (as  far  as  we  Icnow)  used  two  dimensional 
orthogonal  polynomials  (searching  for  their  common  zeros)  and  poly¬ 
nomial  ideal  theory,  see  [1i]  for  an  extensive  set  of  historical  and 
current  references.  We  were  inspired  by  the  operator  theoretic  treat¬ 
ment  of  moment  problems  as  developped  in  [12],  see  al^o  the  method  of 
the  paper [16]. 

*  on  leave  from  Dept,  of  Niiner.Anal.,EdtvOs  University 
H.  1088, Budapest,  Muzeun  k.6-8,Ffie'p. 
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Since  the  minimal  support  solutions  are/  in  general  non  unique 
in  the  higher  dimonslonal  case  (in  contrast  to  the  onedimensional  case) 
moreover  their  set  (thus  the  problem  of  i’nding  at  least  one  element  of 
it)  is  not  convex  ard  for  other  reasons  li)te  the  complexity  and  stab¬ 
ility  (  with  respect  to  errors  in  the  prescribed  moments) we  propose 
and  study  here  an  other, particular  (nonmlnimal)  solution  ,  l.e.  iiiass 
distribution,  the  so  called  analytical  centre  of  the  feasible  set  (of 
solutions).  Several  positive  features  and  applications  of  this  solu¬ 
tion  concept, li)ce  stable  computability  with  a  relatively  small  number 
of  arithmetical  operations  and  the  feasibility  of  high  degree  homo- 
topy  methods  for  computing  bounds  for  any  further,  not  specified 
"moment"  (i.e.  integrals  with  respect  to  the  underlying  measure)are 
studied  in  the  last  section. 


2.  Preliminaries 

Suppose  that  S  c  is  a  cloaod  set  and  y  is  a  nonnegative  (Radon) 
measure  supported  within  S.  In  the  general,  finite  or  restriced 
moment  problem  we  shall  study  here  the  data  are  the  N  values  reals 

(2.1)  C.  =  ;  K.  (s)  p(ds)  =  ip.  (ji),  j=1,...,U 

J  S  1  1 

of  fixed,  linear  (continuous)  functionals  ipy  given  oy  continuous  on  S 
functions  K^,  j  =  1,...,N  on  S  and  one  aslts  for  thi'  conditions  of  the 
existence  and  a  characerization  of  all  solutions  y  which  have  minimal 
support  belonging  to  S; 

(2.2)  M  min,  =  I  (Sj^)  £  O,  Sj^€  S,  k  =  1,...,M. 

2 

In  the  case  when  S  c  R  ,  i.e.  n=2,  and  for  S  =  (x,y)  the  functions 
K, , . . . ,K„  have  the  form 

1  N 

(2.3)  x^y^,  (i,j)  6  I,  III  =  N 

2 

where  I  is  a  finite  subset  of  (  the  set  of  nonnegative  entlres)of 

cardinality  N,  the  above  problem  -  the  so  called  restricted  polynomial 
moment  problem  -  is  a  natural  generalization  of  the  Gaussian  quad¬ 
rature  problem.  Of  course,  one  can  expect  a  reasonably  simple  and 
constructive  answer  to  this  problem  only  if  I  and  S  have  a  simple 
form,  e.g.  S  is  a  quadrangle 

(2.4)  S  =  [a^,b^]  *  ^ 

and  -  for  some  fixed,  positive  L  - 


/ 
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(2.5)  1  ={(i,j)ll  +  j  S  L,  i,j  i  O}. 

Ve  give  now  an  equivalent  foxnulatlon  of  the  problem(2.2)-(2.3) 
which  ia  crucial  for  our  approach. 

Proposition  1 .  The  problem  (2.2)-(2.3)-  with  data  settc(l),s]is 
equivalent  to  the  existence  and  characterization  of  quadruples 
H,A,B,e  ,  where  H  is  a  Hiloert  space  (whose  dimension  should  be  mini¬ 
mized)  ,  A  and  B  are  self  adjoint  cummuting  operators  on  H  and  e  is  a 
nonzero  vector  in  H  such  that 

(2.5)  »<A^B^e,e>,  for  all  (i,j)  £  1. 

Proof.  If  there  is  a  solution  of  problem  (2.2) -(2.3)  then  we  define 
the  Hilbert  space 

(2.6)  H:  =  L2(S,du),  e:  =  1  on  S 
and  the  operators 

(2.7)  A  f(x,y):=  X  f(x,y)  B  f(x,y):  =  y  f(x,y) 

which  are  self  adjoint  and  commuting.  The  conditions  in  (2. 2) can  be 
expressed  as  those  in  (2.5). 

Conversely,  suppose  that  (2.5)  holds  and  let  A,B  have  tlfra  eigen¬ 
vectors  (they  are  common  and  form  a  basis  of  H  by  the  communtativity 
and  self  adjointness  of  A,B)  eigenvalues  x^,..,x^  resp. 

M  is  the  dimension  of  H 

(2.3)Ayj^=Xj^y^,  Byj^  =  y^f^,  J<= 

Then 


(2.9)  c 


ij 


€  I 


where 


Hk 


«<  y  ,k=*1 , . . .  ,M  . 


This  completes  the  proof  and  shows  that  once  we  constructed  the 
quadruple  <H,A,B,e>  then  the  quadrature  formula  (2.9)can  be  obtained 
by  a  low  complexity  stable  numerical  method  l.e. solving  an  eigenvalue 
problem. 

Not  assuming  H  to  be  finite  dimensional  we  had  to  invoke  the 
general  spectral  decomposition  theorem,  see  e.g.  [  12  j  ,by  which  a 
representing  measure  is  obtained  from  the  associated  projector  measure 
dy  (  X)  ■  d(<  E(;^e,e>) 

Proposition  2.  If  problem  (2.1), (2.3)  has  a  solution  then  the  problem 
(2.2), (2.3)  also  has  a  solution, moreover  for  the  minimal  value  M  we 
have  the  inequality 
(2.10)  min  MS  III 

which  is  exact  in  the  sense,  that  there  exist  (multiple  connected) 
domains  S  such  that  for  the  constant  weight  function  y'(x,y):  1  on  S 
and  the  set  I  as  in  (2.5),  for  arbitrary  L  we  have  equality  in  (2.10) 

-  The  first  part  is  known  as  Chakaloff's  theorem  see  [6]  and  is  based 
on  the  simple  fact  that  if 


V 


I 

) 
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R  k 

C  =  Z  y,e.,c  €  i  0,i=1,...,R 

i«1 


1  i 


then  there  exist  a  similar  representation  in  which  there  are  at  most 
k  nonzero  constants  For  a  proof  of  the  second  part  see  §4,ch.2in 
[11].  Before  going  further  let  us  indicate  here  the  connection  of  the 
ed)ove  problem  with  the  mlnir.al , partial  relization  problem  for  a  class 
ot  two  dimensional  shift  Invariant,  linear  input-output  maps 

(2.11)  Yv  1  =  r  F.  .  ,  .  a.  . 

kSi.lSj  3 

by  state-space  models  of  the  form 

(2.12)  =  <h,x^^j^> 


^k+1,1+1  “  ^1*k,l+1  ^2*k+1,l  ■  '‘l®’2*k,l  ^“k,l 

M  M 

where  F-j  >^2  are  commuting, symetric  matrices  in  R  and  h,g  £  R  ,see  [3]. 
The  transfer  functions  assiciated  to  such  maps 


T(w,2)  =jJ' 


dy (X, y) 


(1-wx) (1-zy) 


=  f  ? 

i=0  j=0 


F^.wS^ 


are  generalizations  of  the  one  variable  Stieltj as  functions  and  should 
play  the  same  roie  in  analyzing  “passive"  input-output  maps.  Note  that 
the  realizability  conditions  have  the  form  of  complete,  infinite  moment 
conditions,  if  g  =  h, 

F^^  .  =  <h,F^  F^  g>  ,  i,j  a  O. 

It  is  known  that  the  minimal  partial  realization  problem  underlies  most 
of  the  basic  engineering  problems  of  system  analysis,  see  e.g.  [2j, 
even  if  for  a  Slliiable , more  exact  and  stable  numerical  solution  of 
these  problems  other  linear  information  functionals  are  better  suited, 
see  [14]  and  below. Connections  to (rational) approximation  (interpolation) 
problems  for  Stieltj es  functions  are  extensively  studied,  see  e.g.  [7], 
[10], [14], [16]. 


3 .Exact  conditions  of  existence  and  minimality 

He  shall  restrict  our  interest  to  so  called  "regular "index  sets  I, 
which  -  by  definition-  have  the  following  property. 

(3.1)  if  (i,j)  €  I,  then  (k,l)  £  l,  for  all  k  S  i,  1  S  j 

In  order  to  characterize  the  minimal  solutions  (H,A,B,e)  we  have  to 
characterize  first  the  sets  with  conslts  of  a  maximal  number  of 
linearly  independent  vectors  among(3.2)  A^B^e,  i,j  a  0. 

Lemma  1  In  the  linear  space  H  spanned  by  the  vectors  (3.2)  (if  it  is 
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finite  dimensional)  there  always  exist  a  basis  consisting  of  ele- 

2 

niehts  of  a  regulu  sdbset.  L  c  . 

Proof.  Let  n^  be  the  meucimum  of  the  values  n  such  that  e,Be/ .  ■ 

are  linearly  independent.  Suppose  inductively  that  n.  ,k  i  1  is  the 
largest  value  of  n  such  that  B  A  b  is  linearly  independent  on 
tne  vectors  A^B^e,  with  i  i  k-2,j  S  n^  and  i"k-l  js  n-2. Since  the 
sequence  of  the  n^^fk-l .satisfies  n^2  n^...^ 
the  above  procedure  ends  in  at  most  dim  H  steps  and  -yields  a  regular 
set  L. 

Definition.  If  L  is  a  regular  set,  the  (generalized) Hankel-matrlx  asso¬ 
ciated  to  it  is  defined  by 

where  we  order  the  rows  and  colums  of  (indexed  by  elements  of  L) 

2 

according  to  the  lexigographic  order  in  Z_^.  Further  we  denote-for 
a  regular  set  L 

L*;  =  {(k,l)  I  3  (i,j)  €  L  with  1  2  k-i  2  0,  1  2  1-j  2  0} 

L^:  =  {(k,l)  |3(i,j)€L,k  S  i-H  ,l=j  )  ,L2-{  (k,l)|  3  (i,  j)€L,k=i,lSj-M  } 

L^;  =  {  (k,l)  (k«i^-H2»l=j^+j2»  (i^  »  j^)CL,  (l2» 

Theorem  1 .  The  necessary  and  sufficient  condition  .  for  the  existence  - 
given  the  moment  data  c(I)  -  of  a  nonnegative  representing  measure 
supported  in  at  most  M  points  of  s  is  that  there  exists  a  regular 
set  L  of  cardinality  m  and  an  extension  of  the  data  from  c(I)  to 
c((L*)^),  i.e.  an  assignment  of  values  to  the  unspecified  moments 
in  c((l'‘)^)  such  that  the  matrix  is  positive  semidefinite  and 
(3.3)rank  =  rank  SM. 

Moreover  the  minimal  value  of  M  for  which  the  above  two  conditions 
can  tie  satisfied  equals  the  minimal  number  of  knots  in  the  corres¬ 
ponding  cubature  formula. 

Proof.  In  order  to  understand  the  role  of  the  matrices  and  note 
that  these  are  theGram  matrices  associated  to  the  set  of  vectors 
W(L)  -  { A^B^e  I (i,j)  €  L) 

W(L*)  -  (A’^B^vIv  €  W(L),  0SrSl,0isSl}. 

The  necessity  of  the  conditions  (33)  follows  now  from  Proposition  1  and 
Lemma  1  since  Gram  matrixes  should  be  positive  semidefinite  and  their 
rank  equal  the  dimension  of  the  space  spanned  by  the  underlying  vectors. 
To  prove  the  sufficiency  of  the  conditions  we  have  to  construct  a  quad¬ 
ruple  (H,A,B,eJ  ,  such  that  dim  H  >  rank  and  (2.5)  holds.  Now  we 
define  H  as  the  Hilbert  space  spanned  by  vectors  indexed  by  the 
element  (i,j)  €L*  ,  whose  scalar  products  are  specified  by 


‘\2  ■>/  I  nj^«dim  H 
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Since  rank 
AV 


®i+k, j+1 


i.j  ’i+1rj  "'i,j 
defined  on  the  whole  space  H,  moreover  they  are  well 


'V,  , ,  ^  BV 

T  .  T  ill  ^ 

are  herety 

defined:  if 

V  »  Z  a.  . 
r.s  (i,j)€L 

for  all  (k,l)  €  L*  ,  then 


(l,j)€  L 


/i  j,  i.e.  z 


^r+1,s  7i,f)€L 


®i,j  '^i+nj  ''r,s+1 


>  Z  a 
(i,j)€L 


k,l' 


hold.  Indeed  multiplying  the  latter  relations  by  Vj^  ^,(k,l)€  L,  the 
relations  obtained  are  cohsequences  Of  the  previous  ones  because  is 
a  submatrix  of  H..  and  H.. 


and  these  are  submatrixes  of  * . 

Lj  L 


These  operators  A  and  B  are  clearly  symmetric  (i.e.  self  adjoint) 
since  for  all  (i,j),(k,l)€I. 


<  *'^i,j'\,l>  = 


c.  .  ,  .  ,  =<  V,  .,AV,  .  > 

i  +  1+)c,3+l  i,j'  k,l' 


and  they  commute,  since 


^*®^i,j'\.l>  = 


‘^i+k+1 ,  j+1+1  j'\,l^ 


By  this  the  theorem  is  proved. 

The  difficulty  with  this  extension  problem  is  partly  apparent  from  the 
following  fact: the  restriction  of  th«f  original  say  infinite  dimensional 
operators  A  and  B  to  a  Krylow-like  subspace  W(L)  are  symmetric  but 
they  may  not  commute,  (in  general,  they  do  not  commute)  It  is  not 
clear  what  further  connections  (if  any)  exist  Ijetween  the  set  I  (and 
the  values  c(I))  on  one  side  and  the  possible  sets  L  on  the  other  side, 
is  it  true  that  L  can  be  chosen  as  a  subset  of  I? 

These  sharp  differences  between  one  and  higher  dimensional  polynomial 
moment  problemshave  been  observed  e.g.  in  [13], where  it  is  first  shown 
that  in  the  twodimensional  trigonometric, finite  moment  problem  the  non- 
negativity  of  the  associa.ted,  generalized  Toeplitz  matrix  (the  precise 
analogon  of  our  Hankel  matrix)  is  not  sufficicient  for  the  solvability. 
The  theory  of  normal  extensions  of  operators, see  the  appendix  written 

by  SzSkefalvi  Nagy  in  [12], is  clearly  related  to  our  problem  since  the 
operator  A  t  IB  >  T  should  be  normal,  for  A,B  to  be  symmetric  and 
commuting  and  vice  versa.  The  conditions  -  in  terms  of  c(I)  -  for  the 
condition: spectrum  T  c  S  can  be  easily  written  down  in  the  case  (2.4): 
the  following  matrixes  should  be  nonnegative  definite 
(3.4)  H,  -  a,Hi  ,  b,H^-H,  ,  bjH^^  -  H,  , 


■l"l 


1“L  “L, 


"1  .  *  I  «  ilj  U  ilj 

If  S  is  the  disjoint  union  of  two  quadrangles  and  (32  than  we  have  to 

require  that  there  exist  a  decomposition  of  each  of  the  moments  (fixed 


323 


or  assigned)  such  that 

®i/j  “  ®ij  ^ 

and  (3.4)  holds  for  the  respectively  decomposed  matrixes. As  an  example 
of  a  simple  application  of  Theorem  1  we  metlon  the  following  fact: 
fir  six  data  (c_  _;c,  ;c^  ,)  If  the  coresponding . 3  x  3  mattix 

Is  nonsingular  the  minimal  measures  should  have  3  atoms  and  they  con¬ 
stitute  a  one  parameter  family. 

A  new  numerical  approach  to  solve  the  existence  problem 

It  is  very  difficult  to  handle  the  Constraint ( 3 . 3 )  numerically,  the 
set  of  solutions  of  the  minimal  extension  problem  Is  not  convex. 
Observing  that  the  finite  dimensional  analgon  of  the  solution  set  to  a 
moment  problem  (2.1)  has  the  form  of  a  polyhedron  (  in  the  sequel  we 
often  use  abbreviations  for  N  tuples  (c^,...,Cj^)  =  c**) 

(4.1)  K  =  K  (k^,C*^)  =  {  ^  |<  c^^  1=1, ...,N,  e  } 

we  see  that  searching  for  the  extremal  points  "vertices"  of  K. 

It  is  known  that  the  parameters  of  a  Gaussian  quadrature  are  very  111 
conditioned  functions  of  the  moments  (  note  that  (2.1)  Is  something  like 
an  Integral  equation  of  the  first  order  whose  right  hand  side  is  known 
only  at  some  points)  -  and  this  has  its  parallel  in  the  fact  that  the 

M  M 

vertices  of  a  polyhedron  H(k  ,  c  )  are  nonsmooth  functions  of  the  data 
gN,  or  (k^,c**). 

We  propose  now  using  an  other, specific  solution, tae"analytic  centre" 
of  the  solution  set,  in  order  to  solve  the  existence  (  and  some  related 
estimation)  problems,  in  a  numericaly  more  feasible  manner. 

The  analytic  centre  p  (K)  «;»(k“,c*’)  of  the  polyhedron  ( 4 . 1 )  is  defined 
as  the  unique  point  which  solves  the  following  optimization  problem 

max{^?^lcg  Cj  •  j®!  *  •  •  a  0,i»1 , . . .  ,m} 

If  the  polyhedron  is  represented  in  its  own  space (of  dimension  m-M,in 
general),  l.e.  K  -*  P  ■  P(a***,b'’') 

P(a"*,b"*)  -  (xlb^-<a^,x>  i  0,  1-1,..., m,x  6  r’"'*’} 

by  the  map  -  bj^-<a^,x>  ,i-1,...,m,  then  jt  *  x  (a®,b®)  the  point, 

which  solves  the  problem  (assuming  int  P  *  0) 
n 

max{  JI  (b,-<a,  ,x>)  lx  €  P(a"*,b“)) 

1-1  ^ 

One  can  prove  that  the  map  (a^jb™)  -»  x(a®,b®)  is  effihe  invariant 
and  there  exists  a  two  sided  ellipsoidal  approximation  for  P  around  x: 

X  +  E  c  PCX  +  m  E,  E  -{  zl<Az,2>  s  1} 


/ 
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where  the  symetric  matrix  E  «  E(a®,b®)is  easily  obtained  from  x(a®,b®) 
see  [14],  [15].  The  fact  that  x(a”',b®)  =»  n(k**,c**)  is  an  analytic,  very 
smooth  function  of  the  data  allows  to  solve  the  feasibility  and  linear 
optimization  problems  by  a  homotopy  approach, see  [15],  which  we  gene¬ 
ralize  now  as  follows. 

The  analytic  entre  of  the  set  (2.1)  is  defined  (if  its  exists)  as 
the  solution  of  the  problem 

(4.2)  sup{/  log  u'(s)dsl  /  K.  (s)tt'(s)ds  =  c.,  j“1,.,.,N}. 

S  S  jj 

It  is  easy  to  prove  that  the  set  of  values  c  for  which  (4.2)  has  a 

N 

solution  IS  convex  and  dense  in  the  set  of  all  feasible  c  ,  if  S  is 
a  domain,  i.e.  closure  (int  S)  =  S.  For  the  trigonometric  moment  pro¬ 
blem  this  solution  was  studied  already  about  1920,  see  [10], [14]. 

Lemma  1 .  The  solution  of  the  problem  (4.2)  -  if  it  exists  -  has  the 
following  form  ^ 

j.'  (s)  =  (  I  a.  K  (s))"'' 
j=1  J  J 

for  suitable  which  in  fact  is  then  the  unique  solution  of  the 


equation 


N  .1 

J  K.  (s)  (  Z  a.K. (s))  'ds  =  c.,j  *  1,...,N 


such  that  z  OjKj (s)  is  positive  on  S,  here 


(4.4)  F(a)  =^/  log  (  Z  ajKj(s))ds 

Proposition  the  moment  problem  (2.1)  has  a  solution  if  and  only  if  the 
iJ  N 

homotopy  path  a(X)  can  be  continued  from  X =  1  till  X®  0,  where  a  (X) 

N 

0  <  X  i  1  is  defined  as  the  solution  of  (4.3)  where  c  is  replaced  by 

(  1  -  X)C^  +  , 

Co  =  /  k'^Cs)  (  I  o’k  (s))^ds 
N  o  S  j=1  3  ^ 

and  E  o,  K.  is  an  arbitrarily  fixed  polynom  which  is  positive  on  S. 
j=1  ^  ^ 

The  proof  is  a  simple  application  of  the  impllcite  function  theorem. 
For  brevity  we  can  only  refer  tB[9 ] , [15] for  the  application  of  this 
method  for  the  estimation  of  (computation  of  exact  upper  and  lower 
bounds  in  terms  of  the  moments  c  for 

l(c”)  i  I  K^(s)  u  (ds)  s  u(c*‘) 

It  can  be  expected  that  for  smooth  analytic  kernel  functions 
K^,K|,...,K^  this  approach  is  superior  to  those  using  discretizations 
of  the  measure  (of  the  set  S)  and  algorithms  based  on  the  simplex 
method  (note  that  the  latter  methods  use-  as  a  tool  -  extremal  solu¬ 
tions,  only  piecewise  smooth  homotopies) {concerning  numerical  test 


/ 
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results  on  this  approach-using  homotoples  along  analytic  centers-  to 
solve  linear  programming  problems,  see[9]. 

The  special  solution  of  (4.2)  In  the  case  of  the  (real) trlgono  - 
metric  moment  problem  -  where  K^(s)  «  exp(l ( j-1 ) s) ,  s  e[-n,n]and  u  a 
measure  onl-n, (which  Is  symetrlcal  to  zero)-, which  Is  a  special 
case  of  the  Nevanllnna-Plclc  moment  problem.  Is  the  so  called 
"maximum  entropy"  solution.  These  analytical  centers,  more  precisely 

the  coefficients  of  the  trigonometric  polynomial [  y' (e^®)]”^  »re  ratio- 

N  2 

nal  functions  of  c  which  can  )3e  computed  rather  qulc)cly:ln  0(tl  ) 

arithmetical  operations.  This  and  other  observations,  see [15], lead  to 
the  Idea  that  for  the  extrapolation  of  the  function  a^(x) rational 
(multipoint  Fade)  approximation  -  with  Newton  type  corrector  step  to 
solve  (4.3)  -  will  furnish  a  rather  efficient  path  following  method. 

In  fact.  In  a  problem  closely  related  to  (4.2),  the  use  of  a  special, 
rational  extrapolation  method  can  be  justified  rigorously  using  a 
generallzalton  of  the  well  Icnwon  fact  (see  e.g.[7])  that  the  multi¬ 
point  Fade  approxlmants  (1 .e.lnterpolants)  to  a  Stleltjes  function 
are  again  Stleltjes  functions,  see[151. 

In  order  to  solve  -  over  some  domain  S  -  the  closely  related  uniform 
approximation  problems 

minll  K  (s)  -  §  p.K.  II 

gN  °  1=1  ^  ^  ^ 

'  N  * 

we  propose  following  the  homotopy  path  5  (]()  determined  by 

sup  (log(X-e)  +  /(log  (K  (s)-  Z  B.K.  (s)-e)+  log  (e-K  (s)  + 

(e,(,N)  S  °  1=1  ^  ^  ^  ° 

Z  B.K, (s) ) )ds  . 

1=1 

Of  course, the  sucess  of  these  methods  depends (among  others)  on  the 
availability  of  fast  and  accurate  methods  for  approximating  the  above 
Integrals  as  well  as  those  In  (4.3). 
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ABSTRACT 


The  aim  of  this  paper  is  to  propose  an  approximation  procedure  to  compute  tire  value 
function  V  and  the  optimal  policy  u  related  to  the  stochastic  problem  (P)  of  controlling 
diffusion  processes.  This  procedure  can  be  easily  extended  to  problems  for  which  stopping 
time  and  impulse  controls  are  also  considered. 

O  -  INTRODUCTION 


As  we  did  in  (8]  for  deterministic  problems  we  will  employ  here  as  basic  tool  of 
analysis  the  characterization  of  V  as  the  maximum  element  of  a  suitable  set  W  of  functions  w. 
While  in  [8]  the  definition  of  Uf  requires  for  w  to  be  subsolution  of  the  first  order  Hamilton - 
Jacobi 'Bellman  equation,  i.e.  : 


(0.1) 


here,  in  the  stochastic  case,  we  deal  instead  of  (1)  with 

L(u)w  ♦  l(u)  >  0  (0.2) 

where  L  is  a  second  enter  differential  operator. 

In  what  follows  (P)  will  be  solved  using  the  characterization  mentioned  above.  To 
introduce  tire  discretized  problems  (P**)  we  treed  to  define  properiy  the  functions  wh  belonging 
to  In  fact  :  the  existence  of  maximum  solution  V**  for  each  problem  (Pl*)  and  the 
conveigence  of  Vh  to  V  are  shown  using  a  Discrete  Miudmum  Rrindple  (E^dP)  that  wh  must 
verify  (cfr.  P]).  To  insure  this  property  we  use  particular  schemes  to  discretize  the  first  and 
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second  derivatives  of  w.  FuithennKe  this  choice  enable  us  to  compute  Vh  using  an  algorithm 
of  relaxation  that  increases  the  values  of  in  the  veitices  of  the  triangulation  employed 

G)mments  on  applications  are  included  in  the  final  chapter. 

1  -  THE  PROBLEM  tPl 

Let  us  consider  ; 

a)  The  complete  probabilistic  space 

(Q.P,  F.  nO);  (l.l) 

b)  The  state  process  y(.),  modelled  by  the  diffusion 

dy(t)  =  f(y(t),  u(t))dt  ♦  o(y(t),  u(t))  dw(t) 

(1.2) 

y(o)  =  X  ,  t  >  0,  y  €  Qc  R" 


with 

Q  :  open  boundet  set 

w(t)  :  Wiener  process  F(t)-measurable 

u(t)  ;  control  process  progressively  measurable  in  a  compaa  set  U  C  R™ 
o  is  a  n  x  n  matrix 

f  and  <3  bounded  continuous  on  Q  x  U. 

c)  The  cost  functional 


J(x.u(.)) 


I(y(s).u(s))  C-*  ds} 


(1-3) 


with 

X  :  fiist  exit  time  of  Q  of  the  system  trajectoy 
a  >  0 

f  ;  bounded  continuous  functicn  on  Q  x  U. 
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Let  us  introduce  the  definition  of  the  optimal  cost 


V(x)  -  inf  J(x,u(.)), 
ueU 


V(a)  being  solution  (cfr.  [S]^])  of  the  Hamiltoa-Jacobi-Bellman  equation 


min  {L(u)V  ♦  l(,u)}  -  0  in  Q 
U€U 


V  =  0in3Q 

where  the  differential  operator  L  is  given  by  : 

“  r.s  =  1  %  I  ^  “ 


ats-i  Z<te<^B.i-e-aB  =  a5r. 
1 


As  it  was  said  in  the  Introduction  we  will  compute  V  taking  advantage  of  its 
characterization  as  maximum  element  of  a  suitable  set.  i.e.  (cfr.  [6],  [8],  [15])  solving  the 
following  auxiliar  problem  (having  V  as  solution)  ; 

(P)  ;  End  the  maximum  elernem  w  of  the  set 

W  -  {w  c  /  L(u)w  ♦  I  >  0  in  lr((2)  Vu  6  U,  Q  C  R"}  (1.8) 


w  <  w  «»  wOt)  <  w(xX  Vx  c  Q 


the  natural  partial  order  in  W. 


(CJuestions  concerning  exisfcnoe  and  unkity  of  the  sohdion  of  (P)  can  be  seen  in  [4],  [15]). 


We  wi!l  compute  V  as  the  limit  of  the  solutions  of  a  sequence  of  approximate 
(xoblems  (P**). 

To  sinqiliiy  the  presentation  we  will  suppose  that  Q  is  polyhedric.  We  consider  in  Q  a 
triangulalion  (union  of  sinqilices),  being  (i  »  1,  2,  Nh)  the  vettices  of  Qf'. 


Then  we  define  W**  by  functions  w**  verifying  properties  related  to  (1.8),  (1.6).  The 
main  difficulty  of  this  iqiproach  is  to  ensum  the  existence  of  a  maximum  dement  wb  m  U**. 

Following  what  we  did  in  [8]  for  the  detenninistic  case  we  introduce  in  Ul**  the  natural 
partial  order 

wj  <  «s»  wjfxJ*)  <  w^fxf),  Vx|*  vertex  of  C?  (2.1) 

We  consider  functions  w**  ;  Qh  _  wh  continuous  in  Qb  with  constant  in  the 

interior  of  each  siiiqilex  of  Qb,  i.e.,  wb  ate  linear  fmile  elements.  So,  to  define  wb  it  will  be 
enough  to  precise  the  inequality  C’discietization”  of  L(u)w  *  t  >  0)  to  be  verifyied  at  each 
vertex  xj*  of  Qb.  Taking  [8]  into  account  if  suffices  to  propose  a  suitable  discretization  of 

^  d^W 

L(u)w  -  ^  ^  ^  the  tenn  containing  Ac  second  order  derivatives  of  w. 

ZZ  Definition  of  Lb(u)  wb 

Let  us  consider  Sfaj*)  (see  Fig.  2),  all  the  simplices  having  as  vertex. 


From  (1.7)  the  matrix  A  >  (an)  has  no  negative  eigenvalues  Xp  and  orthogonal 
eigenvectors.  So 
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with 


A  -  UDir 
UU*  -  I 

D  (diagonal)  /  •  Ap  >  (X 


If  we  consider,  with  center  in 


(2.2) 


transfcxmation  matrix  G(Xj ) 


n  -  G  .  4  (2.3) 

and  we  define 

w(x)  -  w(x|'  ♦  9  t  ^  “  w(G*>n)  ^  «<ii)  (2.4) 


we  obtain 

•*  h  ■'^iv  ••  h  32a, 

U.).  - ^  ,  ««?'■>  ^  ■  .  «■« 

with  bpq(x|’,  u)  ■  (GAGOpq. 

So,  after  the  choice  G  >  IT  we  have,  becuae 

■  Ap  ^  (2.6) 

the  following  diagonal  fonn  of  L  : 

fl  h  ^  ikf 

Lw  -  2.  (2.7) 

P-1  attp 


/ 


(2.1.1) 


♦  Ku.  xf)  -  ^  ^  -  N  “eip)  -  2  wiKxf) 

w*KbJb  - 

♦  w'‘(x!*  ♦  hi  7j|p)  - - U - jj -  I  f(x|*)  I  -  a  w(x?^  +  Ku,  x|b. 

I  bi  -  I 


Hnally  we  can  consider  the  discretized  problem  (P)l>  ;  Hnd  the  maximum  element  wl> 
of  the  set  U**  with  respect  to  the  penial  order  (2.1),  Le.  find  wh(x)  such  that  whfxj')  >  wCxj^, 

VxJ*  e  Of*,  Vw^’e  W.” 


3  •  SOME  REMARKS  ABOUT  w^fx) 

As  Cip,  C^p,  hj*  are  convex  combinations  of  the  vertices  of  SCxj*),  using  the  linearity 
of  wh  we  have  ; 

whfCjlp  *  wh(q^  -  S  >  a  I  ^  -  2  (3.1) 

jef  jef 

ij*  set  of  index  such  that  e  Sfaj*) 

wh(bf)  -  Z  Vi  M^x/*)  >  0,  I  t5  -  1.  (3.2) 

jef 

After  (2.11),  (3.1)  and  (3.2),  we  can  rewrite  Lb(u)w>>(x|^  <■  ((u.  >  0  as  ; 


/ 
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<  ^(x|*.u)  ( X  A^xJ’.u)  X  Tj,- 

lb;  -xj  I 


with  pI'(xJ’.u)  =  [  X 

P“  1  hf 


Xp  (;s^A*) 


I  fl[xS*)  I 


lbs" 


o]-i  >  a 


(3.3) 


Taking  into  account  that  all  the  factors  that  multiply 
(3.3)  are  non-negative  we  can  easily  prove  (see  [8])  : 


in  the  second  member  of 


THEOREM  1 

‘Ihere  exists  an  unique  wh(x),  maximum  clement  of  W^  i.e.  (P**)  has  an  unique 
solution. 


Furthermore  the  operator  L**  verifies  the  following  Discrete  Maximum  Principle 
(DMP)  ; 

(DMP)  :  If  C  is  a  subset  of  vertices  of  Q**  satisfying  L*'(u)  >  0, 

Vxl*  e  Of*,  Vu  e  U*',  there  exiss  F,  0  <  F  <  1  such  that  :  (3.4) 

whfxl*)  <  F  (  |jnax  ^  (whfxj’))  v  0). 


We  can  use  this  DMP  to  esMUish  two  impotant  properties  of  w**. 

The  first  one  is  that  w^  is  characteiuad  by  the  bet  that  (33)  becomes  an  equality  for 
all  xl*  e  (]b  fcr  seme  u  c  IP  when  we  put  ^  mnead  of  w**.  This  charactetizaiion  allows  us 

10  cooqMiiB  ^  using  iterative  algodthms  of  the  same  type  than  those  pesented  in  [8].  The 
value  of  u  giving  the  equalify  will  be  used  to  defitre  the  optiinal  ooottol  fl|i- 


^  'il-  ^ 


A 


The  seooi4  one  ooncems  die  conveiyoce  of  k>  V.  We  have 
THEOREM  2 

The  solutions  ^(x)  of  die  approximate  pioblenu  (P**)  converge  uniformly  to  V(x), 
soluticn  (P),  Le.  : 

lim  I  V(x)  -  ^x)  I  -  0,  Vx  6  Q  (3.5) 

W-0 

where  N  is  the  maximum  of  the  tiiamelets  (tf  the  sin^lex  of  (^.  (see  [8]). 

The  proof  is  achieved  in  two  steps.  We  wiU  briefly  give  here  the  main  ideas. 

In  the  first  part  we  show 

lim  w**  >  V.  •  (3.6) 

IhRO 

For  that  we  regularize  die  elements  of  (1.8)  by  means  of  a  convolution  with  a  function 
of  O*  (R2)  having  a  parameter  p  >  0.  These  fimctions  Wp  can  be  approximate  by  functions 
Wp^  with  this  property  :  the  linear  finite  dement  w^^,  takuig  the  same  values  of  wp,ain  the 

vertex  of  the  oiangulation  Q**,  bdongs  to  W**.  So, 

w"  >  wj„  (3.7) 

If  we  consider  in  (3.7)  the  lower  limits  for  M  »  0^  then  the  limits  for  (p,a)  ->  (0,0),  we 
obtain 


idSo 

Fhialty,  as  w  is  an  ariMniy  dement  of  Uf,  (3.6)  is  ptowed 


(3.8) 


The  seoond  pan  is  devoted  to  show 


We  consider  a  sequence  ot  tuxiliar  psoblem  Pa  for  which  foe  conods  Un  can  take  in 
(1.8)  a  finite  number  of  values  and  foe  number  of  swiidis  within  that  set  d  values  is,  at  most, 
a  If  V„  is  foe  sobidon  of  Pn  ^  can  show 


Vi  >  ^  >  Va  >  9  V- 

IL-ii  V„  -  V. 
n—«o 


(3.10) 


On  the  other  hand  we  consider  the  discretized  problem  for  which  we  prove 


wjj  -  V„  (3.11) 

Ihl—O  • 

^n  >  ^n+1  >  ~  Vn-  (3.12) 


So,  Um  w*'  <  Vn  ;  then,  using  (3.10)  we  obtain  (3.9).  Finally  (3.6)  and  (3.9) 

Ihl-O 


give  (3J). 


The  idea  of  solving  optimal  control  problems  computing  foe  maximum  element  of  a 
suitable  set  of  subsolutions  of  the  Hamilton- Jacobi-Bellman  equation  has  been  recently 
applied  to  sevenl  problems.  Remaining  in  foe  delenninistic  approach  we  have  study  in  [9]  foe 
optiinizaiion  of  an  electricity  production  system  which  comprise  thrae  hydraulic  plants  (two  of 
pumped  type)  and  seven  thermic  plants  (one  nuclear,  two  of  coal,  tow  of  fuel,  one  gas 
powered  and  one  external).  The  numerical  data  have  been  provided  by  EOF  (Electricity  of 
France)  :  they  describe  a  forecast  of  foe  Hrench  system  for  a  wade  of  the  year  2000.  Other 
appUcatioa  can  be  teen  in  [12]  where  several  serial  production/inventory  systems  are 
optimized. 


Concerning  the  stochastic  approach  we  can  mndon  : 


a)  [11]  devoted  to  the  optimization  of  the  syitem  preaented  in  [9]  coosideiing  random 
pewurbations  in  foe  demand  ; 


b)  (7]  in  which  the  ilgoritfam  proponed  m  [10]  for  L(u)  «  A  is  used  to  obtain  the  optiinal 
control  of  a  bidimmsional  diffitsioa  ; 


c)  [1]  in  vriiich  the  numerical  solutkn  of  an  optimal  coaection  proUem  for  a  damped  random 
tineaf  dscOafor  is  studied. 

Hist  appBcations  of  the  procedure  just  proposed  in  $2  and  $3,  as  well  as  a  comparison 
of  these  results  with  those  obtained  by  other  clasic  methods  [13],  [14]  and  [17],  will  be 
presented  in  a  special  session  of  the  next  1EEE>CDC,  Austin,  7-9  Dec.  1988. 
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The  purpose  of  an  electric  power  pool  is  to  reduce  the  cost  of  generating 
electricity  by  transfening  dectric  power  between  the  power  plants  that  are 
controll^  ^  in^vidual  decision  nukers.  During  high  local  demand  a  system 
can  transfer  oiergy  fiom  the  network  into  the  system  and  thus  achieve  cost 
savings.  At  the  same  time  otto  sysrems  produce  electricity  into  the  network 
such  that  power  balance  in  the  netwoik  is  met  The  benefits  of  receiving 
energy  in  one  period  ate  then  conqtensated  by  an  energy  transfa  into  the 
netwotk  in  some  other  period.  As  a  whole,  the  the  pool  can  thus  achieve  cost 
savings. 

The  problem  of  equitable  sharing  of  the  benefits  of  cooperation  during  the 
planning  horizon  is  a  bargaining  problon  in  the  dyn^c  munewoik.  We  shall 
frxmuIiM  the  energy  bargaining  model  in  the  dynamic  framework  and  prcqiose 
a  new  way  of  dividing  the  cost  savings  widiin  die  power  pooL  The  energy 
exchange  contract  is  detennined  (»  die  basis  of  the  Nash  bargaining  scheme. 
In  our  previous  studies  we  have  presented  a  hieracliical  mptoach  to  solve 
Nash  bargaining  problons  in  die  dynamic  framework.  This  approach  is 
extremely  convenioit  in  this  application. 
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The  decomposition  method  has  now  solved  very  accurately  a 
rather  wide  class  of  nonlinear  differential  and  partial 
differential  equations  [1]  shoving  some  significant  advantages 
over  other  methods.  Once  a  problem  is  modeled  with  a  specific 
equation  (linear,  nonlinear,  deterministic,  stochastic,  ordinary 
or  partial  differential  equation)  with  physically  correct  given 
conditions,  the  method  solves  the  equation  without 
linearisations,  perturbations,  closure  approximations,  white 
noise  assumptions,  or  discretization.  Certainly,  much  remains  to 
be  done  on  the  theoretical  foimdation  and  the  precise 
limitations.  Rather  than  a  drawback,  this  is  a  fascinating 
challenge  for  further  work  which  is  beginning  to  be  borne  out  by 
the  increasing  work  in  this  field  particularly  in  Torino  by 
Professor  N.  Bellomo  [2]  and  his  co-workers  [2]  as  well  as  by 
many  others.  The  range  of  problems  solved  and  the  rather 
remarkable  accuracy  obtained  -  the  fact  that  nonlinear  systems 
with  stochastic  parameters  can  be  solved  and  the  fact  that  the 
work  has  applied  effectively  to  parabolic,  elliptic,  and 
hyperbolic  equations  -  certainly  suggest  this  is  a  useful  and 
very  computational  method  for  frontier  applications.  Proof  of 
convergence  and  convergence  rate,  error  estimates,  and  perhaps 
better  generation  of  Adomian's  A^  polynomials  are  fertile  areas 
for  further  study  and  dissertations.  Many  other  research  topics 
are  in  the  area  of  applications;  some  are  discused  in  [3]. 

Let  us  point  out  some  speculations  on  some  Interesting 
possible  future  applications  pointing  out  that  some  of  these 
applications  require  the  development  of  a  correct  mathematical 
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modal  before  decomposition  can  possibly  solve  them.  It  Is  not 
useful  to  apply  the  method  to  many  existing  models  since  they 
have  already  bean  linearized  and  otherwise  simplified  for 
mathematical  tractablllty.  Thus  It  is  up  to  the  expert  in 
physics,  engineering,  biology,  economics,  agriculture,  etc.,  to 
model  the  problems  retaining  the  nonlinearities,  stochasticlty, 
delays,  etc.,  since  the  physically  correct  solution  can  be  very 
different  from  that  obtained  from  the  simplified  models.  Also 
since  the  technique  does  not  require  discretization,  it  is 
evident  that  substantially  less  computing  time  may  be  Involved  in 
a  difficult  problem  such  as  Navier-Stokes  equations  {4]. 

Nevertheless,  some  possible  applications  which  represent  an 
exciting  challenge  are  areas  such  as  nonlinear  and  possibly 
stochastic  and  multidimensional  optiomal  control  theory, 
hypersonic  flow,  quantum  theory  and  gravitation,  generalization 
of  the  Kalman  filter,  and  problems  of  large  space  structures  such 
as  vibration,  heating,  etc.,  [3]. 

Before  going  into  these  areas,  let's  look  briefly  at  some 
illustrative  decomposition  examples  chosen  for  clarifying 
procedure  rather  than  for  difficulty. 


Consider  an  ordinary  differential  equation 
d2u/dx2  -  40XU  -  2,  u{-l)  -  u(l)  -  0.  Let  L  -  d^/to^  and 

write  [1] 

Lu  ■  2  +  40xu 

U  -  Cj^  +  CjX  +  l"^(2)  +  l”^{40xu) 


Let  Uq  ■  cj  +  C2X  +  L“^(2)  -02^  +  C2X  +  x^  and  let  u 
The  components  of  u  are  given  by 


a» 

S  «n- 

n»0 


’^n+l  -  I*"^40xun 


for  n  2  0  thus 

Ui  -  L'^OXUq  -  (20/3)Cix3  +  (10/3)C2x4  +  2x5 
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Slailarly 

Uj  ■  (80/9)Cj^X*  +  (200/63)C2X^  +  (10/7)X® 

n-1 

We  continue  to  some  n-tern  approxiaation  t'n  *  S  ^*1  which 

i-0 

approaches  u  >  ^  as  n  a  flj.  If  we  write  Pj  as  an 
h-0 

approximation , 


-  Cj^  +  c^x  +  x^  +  (20/3) Cj^x®  +  (10/3)C2X^ 

+  2X®  +  (80/9)Cj^X®  +  (200/63)C2X^  +  (10/7)X® 

Imposing  the  boundary  conditions  at  -1,1,  we  write 
•j(l)  ■  OjC-l)  ■  0  from  which  we  get 


149/9 

473/<3 

=1 

-31/7 

29/9 

-53/63 

m 

-3/7 

from  which  Cj^,  C2  are  evaluated.  Substituting  V],  onto  the 
left  side  of  the  differential  equation,  we  should  get  the  right 
side,  or  2,  if  the  approxiaation  is  sufficient.  We  note  that  the 
12-tera  approximation  yields  2.000000  or  seven-digit  accuracy. 

On  with  Ijjc  ■  a 2/9x2,  Ly  ■  I**  ■  a2/dz2  ^e 

write 

[Ljj  ■  f(x,y,i)  +  k(x,y,z)u 


Solve  for  each  linear  operator  in  turn.  Operate  on  each  of  the 
throe  equations  with  the  appropriate  Inverse  and  write 


u  -  *y  + 

u  -  -  L^^ku  -  l;^l^  +  Ly)u 

where  9  ,  «  ,  «  are  the  homogeneous  solutions.  Adding  and 
X  y  z 

dividing  by  3. 

u  «  u_  +  Ku 
0 

with 

Uq  -  (1/3)  *  *z  *  ^^x^  ■*■  ^y^  * 

K  -  (1/3)  (l"^  +  +  L”^)k  +  L“^(I-y  +  \) ) 


+  L*^(L  +  L  )  +  l”^(L  +  L  )) 

y  '  2  X  z  X  y 


assuming  u  ^  u  , 
n»0  ” 

u  -  K  u 
n+1  n 


so  all  components  are  determined.  The  inverse  operators  are 
doable  integrations  leading  to  two  constants  of  integration  to 
be  determined  by  forcing  Uj^  to  satisfy  the  given  condition. 

Suppose  k  >  k(u)  so  the  equation  becomes  nonlinear.  The 


nonlinear  term  is  expanded  as  where  the  Aj,  are 

nwo 

Adomian  polynomials  Cl>3]  generated  for  the  nonlinear  term  and 
the  procedure  is  as  before  except  that  the  Un.|.x  will  involve  an 
A^  term.  Since  each  A^  depends  only  on  uq.U]^,  . . .  .u^,  the 
solution  can  be  obtained  essentially  as  easily  as  in  the  linear 

ease. 


/ 
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Rather  than  further  diecussion  of  tha  aathodoiogy  on  whi^ 
there  is  now  a  considerable  published  literature  in  the  U.S.  and 
Europe,  let  us  speculate  on  sons  applications  idilch  appear  to  be 
possible  in  the  very  pear  future  although  they  regulre  the 
■odelllng  expertise  of  theorists  concerned,  priaarlly  with  each  of 
those  areas. 

Sobs  of  these,  in  the  authors  opinion,  are 

1)  optiaal  control  for  nonlinear  and,  stochastic,  and  even 
BUltidinensional  systeas, 

2)  hypersonic  flow,  turbulence,  slngle-stage-to-orbit  flight 
essential  for  shuttles  which  can  be  used  for  the 
construction  of  space  stations, 

3)  guantuB  theory  and  gravitation,  and 

4)  generalizations  of  Kalaem  filtering. 

Because  of  page  and  time  liaitations  we  discuss  only  the 
first  two  here. 

1)  SuKKise  we  consider  a  nonlinear,  possibly  stochastic  or  even 
BUltidiaensicnial  systeas  which  we  want  to  control  in  soae  optiaal 
way.  For  a  linear  control  systaa  with  a  quadratic  performance 
index,  of  course  an  analytical  solution  can  be  aads.  Consider 
the  state  equations 


X{t)  *  f(X. ,...,X  7 
*>  n 


?t) 


is,  a  set  of  n  nonlinear  differential  equations  with  x(t) 
representing  a  state  vector  with  n  components  f]^,...,fn,  and 
x(to}  a  given  initial  vector.  Define,  for  example  [5]  a 
performance  functional  J’(x,u,t)  given  by 


J  - F(x,a,t)  dt 


where  e  and  F  are  scalar  functions  with  necessary  smoothness 
properties.  Z,et  p  >  CPxf>«Pn]^  ^  *  vector  of  Lagrange 
Bttltipllers  and  form  an  augmented  functional 

J'  -  etx(t^),t^]  +  tF(x,u,t)  +  p*  (f-i)]  dt 


Integration  by  parts  leads  to 


J'  -  •  -  [P*x]  [H  +  p*  X]  dt 

with  H  defined  as 

H(x,u,t)  -  P(x,u,t)  +  p*f 

if  u  is  defined  on  tg  ^  t  ^  tx  >  ve  vary  u  and  find  the 
variation  4J'  corresponding  to  tu,  leading  to  the  n  adjoint 
equations, 


Pi  "-7^1 

so  we  have  a  system  of  2n  nonlinear  differential  equations  with 
two-point  boundary  conditions.  Although  this  approach  has  been 
discussed  by  R.E.  Bellman  and  many  others  perhaps  most  recently 
in  [5],  analytical  solution  has  usually  not  been  possible  except 
by  numerical  methods.  He  now  have  a  promising  and 
potentially  valuzd)le  alternative  since  such  systems  of  nonlinear 
differential  equations  have  been  solved  (even  for  the  stochastic 
and/or  multidimensional  cases)  in  a  analytic  approximation  by  the 
decomposition  method  [1-3]. 

Another  possibility  is  through  solution  by  decomposition  of 
the  matrix  Riccati  equation  which  appears  in  invariant  imbedding 
and  neutron  transport  theory  as  wall  as  modem  control  theory. 
Consider 

R'(x}  -  B(x}  0(x)R(x)  •••  R(x)0(x)  +  R(x}B(x)R(x) 

R(0)  -  0 

where  B,  0,  R  are  continuous  n  x  n  non-negative  matrices. 
Suppressing  the  argument  x  ,  we  have 


R'-B+DR  +  R0-»-  RBR 


If  I.  -  d/dx 


LR  -  B  HR  NR 

irtMT*  LRaR'  ,HR«DR-t-RO,  and  NR  rapraaants  a  nonlinaar 
oparator  on  R  .  Slnca  R(0)  -  0  ,  oparation  with  on  both 

sidaa  ylalda 

R  -  l“S  +  l"Sr  +  l"^NR  . 

Lat  R  and  NR  ba  written  in  tanu  of  Adoaian's 

polynoadals.  For  R  this  is  aqulvalant  to  writing  R  "  ^  Rn 

n«0 


For  NR  wa  write  2  ^n  ♦  Identify  Rq  «  L"^B  then 
n~0 

%  ”  ® 

R,  -  H  R^  +  l“^  A^ 

10  0 

R,  -  l“^  H  R  +  a, 

2  ■  X  X 


R  -  l'^  H  R_  ,+  l“^  A  , 
n  n-1  n-l 


for  n  A  1  .  The  A,)  for  NR  are  given  by  [1] 

*0  “  *0  ®  *0 

^  -  *0  B  »1  +  \  ■  \ 

Aj  -  Rj^  B  Rj^  +  R^  B  Rj  +  Rj  B  R^ 

A,  -  R-  B  R,  +  R,  B  R^  ♦  R,  B  R,  R,  B  R, 
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(2)  tiw:  Hi*  prMwtit  »ppr6acft  to  hjnp«z«dtile 

flow  Jjt  copputacioniBl  fluid  dyn^  (CTO)  ,  ttid 

iiiti[^i^7.w^  to  diivMoii  «i^ropri|tt«^  coapdtor 

protnEWHi  'fmr  t^o  hypotaOTic  catm^  iri%h  odf^inui^  rapid 
dasB^iegplinto  in  svpareeiip^ter*, aphasia  is  esrtainly 
•I^B^Bpr£ata-  Vat,  anot^^  asthc^logy  nofw  appaars  prad.sing 
idiidi  is  fi^ta  diffwmt  and  rfa'ans  to  hava  a  hi^  potsntial  for 
important  advanti^aa  as  wall  as  a  predsably  high  adaptability  to 
si9sireaqpittara.  tl^s  is  tha  dacosposition  nathod. 

It  yialds  a  npidly  convarging  sarias  solution  in  analytic 
fom.  It  raqoiras  no  linaarization,  parturbation,  closura 
approxiaatim,  or  assuq^ion  of  spaeial  aathaiuitically  tractable 
stoehastie  precgsaas  soCh  as  dalta-corralatad  processes. 

Pr^abiy  aost  important  is  the  fact  that  diseratization  into 
grids  is  unnacMsary.  Banca,  coaputation  should  be  enoumously 
lass,  and  tha  difficulty  of  diffarant  tiaa  scales  in  turt>ulence 
is  avoided. 

In  the  types  of  fluid  flow  which  interest  us,  velocity, 
density,  and  pressure  are  stochastic,  not  constants.  Present 
traataant  of  Haviar-Stokas  eguations  solves  a  simplistic 
aodal,  net  real  bshavior.  Tuzbulanca  is  a  strongly  nonlinear, 
strpngly  stocdiastic  phanoaanon  and  cannot  be  understood  by 
linearized  partui^tiva  traataants.  The  theories  of  physics  are 
perturbative  thaorias  and  tha  theories  of  aathwaatics  are  for 
linear,  opentors  (ot^er  than  soae  ad  hoc  aetbods  for  special 
nonlinear  agnations) .  What  is  needed  is  a  way  of  solving  one  or 
aore  nonlinsar  sboebastie  operatof'  eguatiions  i^ether  algebraic, 
differential,  dalay-differantial,  partial-differential,  or 
systaas  of  su^  agiiatiens.  The  ooaputational  accuracy  of  a 
smpwreeaputar  is  dependent  on  the  sophistication  of  the 
aatheaetlcal  aathods  prograaaed  into  it.  Typical  calculations 
consider  ailllons  of  discrete  tine  interualB  aai  sahll  eneu^  so 
trajeetorios  between  then  can  be  taJcen  as  low-order  poiynoaikls, 
e.g.  ,  gbaksatics.  If  stbchasticity  is  involved,’  taunt  l^te  Carlo 
astiiods  are  tried  tdiich  inserts  randbwness  but  net  tike  prepirly 
eerrelalMd  randoaness  which  is  present  in  the  physical'  inrhblew. 


-•■•t;'-,’  -v.'. 

."'y- 
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In  9«naraliz«d  taydrodynaaies,  th*  foxa  of  Havi*r>Stok«s 
aquations  ia  kapt,  but  tlaa  and  (U>atanea  acalas  ara  Introducad  ao 
ona  CM  go  beyond  contlnuua,  VproxiiMktloh  and  taka  account  of 
aolacular  atruetura.  Bowayar,  applicatlod  to  a  reea  altuation 
bacMwa  aiaply  a  taat  of  tha  validity  of  tba  linaar 
approxiaationa,  aa  polntad  out  in  tha  llt'aratura.  Fluctuatlona 
ara  aa  uai^,  aaauaad  "mall,”  and  dalayad  affacts,  dua  to  tba 
fact  that  raaponaaa  cannot  ba  inatantanaoua,  ara  ignorad. 

Whan  ona  atudiaa  airflow  about  aiixraft  aurfacaa, 
coq^tatlona  ara  oada  tana  of  nilliona  of  pointa,  and  it  is  fait 
that  incraaalng  tha  voluaa  of  coaputation  to  tha  llait  in  an 
ultiaata  axtrapolatlon,  auparcoB^utara  will  yiald  coaplata 
accuracy.  Not  only  doas  thia  ignore  atochaaticity,  it  ignores 
the  sensitivity  of  nonlinear  stochastic  aystaas  to  vary  slight 
chazigas  in  tha  nodal  >  in  fact,  to  changes  assent laliy 
undatarainable  by  aaasureaant. 

To  solve  an  aircraft  problaa  on  contaa^lated  naxt>genaration 
eoaputars,  a  3-'diaanaional  wash  is  ganaratad  which  discretizes 
tha  syatm  of  nonlinear  partial  differential  aquations  into  a 
million,  a  hundred  Billion,  or  perhaps  a  billion  coupled 
difference  aquations  in  as  many  unknowns.  Ona  begins  to  sea  then 
tha  traaendous  data  handling  problm,  tha  nheasaity  -for  izqprovad 
algorithaa,  and  tha  need  for  still  graatar  coaputational  ^pead. 

Na  aay  also  have  aatiy  unknowns  at  each  point,  and,  as  ws  bava 
pointed  out,  tha  syatm  nonllnaaritiaa  and  randoa  fluctuations 
need  to  ba  taken  into  eonsidaration.  Since  usually  solutions  are 
iterative  •  first  solving  an  approxiaation  to  tba  original  systm 
of  differential  aquatiens  and  than  iaproving  tba  solution  by 
rapaatad  substitution  of  each  new  solution  •  parallel  processing 
ia  eo^plieal^  by  the,  Uf f iculty  of  partitioning  tba  work  so  each 
prooessor  earn  work  indepandsntly.  TMs  being  pursued  by  many 
ingaaloas  ideas  nacassitatad  by,  tha  brute  fores  mthed  of 
diseratizati^on. 

In  ail  su^  problaas  wa  need  to  be  abla  to  solva  coupled 
•yataaa  of  nonlinaar  (and  generally  stochastic  aa  wall)  partial 
dlffarantial  aquations  with  eo^plax  boundary  conditions  and 
possibla  dalayad  affects.  Thasa  systaas  ara  linaaritad  and 
diaeratizad  (and  tha  stoatastie  aspects  either  ignorad  or 
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iapr^purly  d«alt  with)  ao- th*  wiaua  BUMrical  a^roxiaatlon 
aathods  can  ba  usad.  This  raqairaa  £aata«  aiMl  fastar 
si^roonputan  to  do<  UMaa  covatations  in  a.  raaaonabla  tina. 

OnfortuBataly  tha  fbrthar  davalqpBnita  in  aupareoapatara  can 
qoita  poaaibi^  giva  wrong  anawata  ^loma  av»n  a  aingla 
ona-diaanaional  nonllnaar  diffaran^lBl  aguBtien  witbout 
atpebaatisitgr  in  coafCiolaatai  in^iuta,.  and  boundary  conditiona-  - 
lat  alona  vaetor  partial  dlffarantla:  a^ationa  in  ^aea  and  tina 
vitJa  Bottlinaax  and/or  ateehaatle  paraaatara  •  ara  not  aolvad 
axaetly.  Raal  ayatana  ara  tionlinaar  and  atoehaatie.  Nhan  you 
throw  out  thaaa  "ccaplieatloM,”  you  hava  a  diffarant  problaal 
Whan  you  llnaariza  aiid  uaa  parturiwtiva  nathoda,  you  aolva  a 
nathanatizad  problan,  not  tha  ^yaical  problan.  Tha  nodal 
aquationa,  avan  bafora  tha  linaarization,  diacratization,  ate. 
ara  already  wrong  bacauaa  tha  atoehaatie  bahavlor  la  generally 
not  incorporated  or  ia  incorporated  incorrectly  aa  an 
afterthought. 

Our  approach  to  hyparaonica,  uaing  daconpoaition,  will  ba 
baaad  on  pravioua  work  on  Naviar>Stokaa  t3»4]  which  ahowed  an 
analytic  aolution  can  ba  earriad  out.  For  hyparaonic  caaaa, 
additional  a££acta  ara  praaant  changing  tha  nodal  aquationa  but 
tha  approach  ia  ainilar.  Diacuaaion  of  a  rathar  global 
nathanatical  nathodology,  lat  alona  tha  huga  sub j act  .of 
hyparaonica  and  tuAulanca  ia,  of  courae,  not  addraaaabla  hara. 

Wa  can  only  call  attantlon  now  to  tha  poaalbility  of  aona 
proniaing  altamativaa  to  tha  praaant  approachaa  [3]. 
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The  paper  provides  a  preliminaiy  exploration  of  the  application  of  fuzzy  arithmetic  and 
fuzzy  approximate  reasoning  techniques  to  qualitative  reasoning  problems  considered  in  Artificial 
Intelligence.  Mote  qtecifically,  this  investigation  is  done  along  three  lines :  constraint  propagation  with 
ill-known  values,  handling  of  orders  of  magnitude  in  terms  of  fuzzy  intervals  or  by  means  of  fuzzy 
relations. 

1  -  Introduction 

Reasoning  about  the  behavior  of  systems  in  a  qualitative  way  is  interesting  in  two  kinds  of 
circumstances :  i)  when  the  system  under  consideration  is  complex  and  the  data  available  about  it  ate 
pervaded  with  ittqnecision  or  even  vagueness ;  ii)  when  it  is  sufficient  to  have  a  qualitative  view  of  the 
system  and  of  its  behavior,  and  this  qualitative  view  is  not  only  easier  to  get  titan  a  more  precise  one 
from  a  computational  point  of  view,  but  also  easier  to  understand.  FIrom  the  beguming  of  the  eighties 
there  have  been  a  growing  interest  about  qualitative  reasoning  in  Artificial  Intelligence ;  see  (Bobrow, 
1984 ;  Dotmoy,  1987)  for  an  introduction.  The  intended  purpose  of  this  research  is  mainly  to  provide 
understandaUe  explanations  of  the  behavior  of  complex  systemt  fiom  their  qualitative  description.  The 
modeling  is  done  in  terms  of  variables  whicb  are  potentially  real-valued,  but  the  analysis  and  tire 
description  of  the  system  behavior  is  made  only  in  terms  of  three  values  usually,  namely  ”0"  and 
corresponding  to  whether  the  variables  are  negative,  zero  or  positive.  Indqiendently,  works 
motivated  by  research  in  qualiutive  economics,  have  been  developed  about  qualitative  controllability 
and  observaMlity  of  linear  dynamical  systems  whelte  real-valued  variables  ate  apptoximaied  in  terms  of 
the  same  three  values ;  see  IVavd  and  Kaszkutewiez  (1986)  fbr  instancs. 

Bram  the  end  of  the  seventies,  fiizzy  set  and  possibility  tiieoiyCZadeh,  1978 ;  Duboit  and 
Ptade,  1985),  whose  introduction  was  initially  motivated  by  tiie  modeling  ooniplex  and  ill-known 
systmis,  hae  been  ooosidMaMy  devektyed  bodi  fkriala  Iheoietkal  and  an  point  of  view  in 
vatioos(8R)cai<ns;p«idciilatly,flatyatitiltnetie0>nboisandFrade,1980. 1987)  enables  us  to  handle 
Ukknowa  ifMtilUdl  in  an  eify  way  which  geneialiaes  haeiTat  anatysis,  arid  beildes  a  naatiiodolagy  ftar 
qpiexiniiatoreaaenbigCBdltntnandZadeh,  1977)  h«  bM  letded  in  tire  fbiay  set  fkatneworiL  Until 
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now  there  have  been  no  serious  attempt  to  use  fiizzy  techniques  in  qualitative  reasoning  probtems  in 
Artificial  Intelligence -if  we  excqit  some  hints  (Raiman.  1985)  and  pieliminaiy  works  (d’Ambiosio. 
1987)- aldiou^  it  would  be  derirable  in  some  cases  »  have  a  finer  and  less  sharp  description  of  the 
values  of  the  variables  than  the  one  provided  by  ‘X)”  or  Particulariy,  the  sign  of  the  difference 
between  two  positive  quantities  cannot  be  determined  widtout  any  informatirai  about  their  respective 
Older  of  magnitude. 

TMs  paper  investigates  what  may  be  the  use  of  fuzzy  arithmetic  and  fiizzy  set-based 
approximate  reasoning  techniques  in  qualitative  reasoning  problems.  First,  a  general  approach  for 
refining  interval  values  attached  to  variables  by  exploitii^  constraints  which  must  be  satisfied  by  these 
variables,  is  extended  to  fiizzy  set  values.  Then,  a  fiizzy  interval-based  qiproach  is  proposed  for 
handling  orders  of  magnitude  in  arithmetic  operations  and  a  valid  approximation  technique  is  used  in 
order  to  insure  a  closure  property  of  the  operarions  restricted  to  the  considered  fiizzy  values.  The 
interest  of  fiizzy  intervals  for  interfacing  symbolic  information  and  numerical  data,  is  emphasized. 
Then  another  way  of  dealing  with  orders  of  magnitude  based  on  approximate  equality  relations  is 
investigated.  The  concluding  remarks  point  out  some  other  contribudras  of  fiizzy  logic  to  qualitadve 
control  and  to  qualitative  descriptions  of  systems  behavim. 

2  .  Constraint  oronaeatlon  with  fuzzy  values 
2J  •  General  Mseussion 

Let  X], ...,  Xn  denote  single-valued  real  variables.  Let  Ai  be  a  subset  of  the  real  line 
which  is  known  to  restrict  the  possible  values  of  Xj,  and  let  R  be  a  relation  which  must  be  satisfied  by 
the  Xi's  and  which  acts  as  a  constraint  on  (Xj, ...,  Xg).  Then,  the  refinement  of  the  possible  ranges  of 
the  variables  Xj's  taking  into  account  R,  leads  to  update  the  possible  range  of  each  variable  Xj  into  a 
new  subset  A'f  in  the  following  way 

A'i-{xi€  Ajiaxju  Aj,j-l,n,j#iand(xi,...,xi,...,Xn)6  R}  (1) 

More  generally  in  case  of  several  constraints  represented  by  relations  Rj^,  k  ■  l,r,  we  can  iterate  this 
refinement  procedure  on  each  variable  taking  successively  each  relation  into  account  over  and  over  until 
no  more  changes  occur  in  the  updated  ranges.  This  is  known  in  Artificial  Intelligence  as  the  Waltz 
algorithm ;  see  Davis  (1987)  for  adetail  study  ^  this  procedure  both  fiom  an  isoplemeniation  and  an 
qtplicatioa  point  of  view.  Let  us  consider  a  simple  exasople.  Let  n  >  3,  A^  >  [0,2],  A2  ■  [13]  and 
A3  ■  [03]  and  the  constraint  Xi  +  X2  -  X3.  Then  we  get  A'|  -  [0,1],  A'2  ■  [13]  and  A'3  -  [13]. 
Observe  that  any  triple  of  values  in  the  Cartesian  product  A'l  X  A'2  X  A'3  is  not  necessarily  feariUe, 
e.g.  $  X3  c  A'3  such  that  xi  X2  ■■  X3  with  xi«landx2*2. 

The  defiititioo  (1)  expresses  diat  A'i  is  obtained  as  the  intersection  of  Ai  with  the  result  of 
the  cooTOsition  of  the  relatioa  R  with  the  Cartesian  product  of  the  i^'s  excqpt  Ai.  This  can  be  readily 
extended  to  die  case  where  dteAj's  are  fuzzy  sets  and/or  Rrepteisoti  a  fuzzy  coosttaint;Le.  . 

Vi,Vxi.  |iA'i(Xi)*ininbiAi(xi).  supxj  ntinO^iCxi,  ....Xg),  ntin  RAjC^)]  O) 

J«l,n;>i  J-l4t:>i 

where  (t  denotes  the  inembenhip  functions  (whose  range  ate  [0,1])  of  the  corresponding  fhzzy  sets 
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and  relation.  When  R  is  an  oidiniiy  relation  such  that  Xi  is  a  fniictx»f  <tf  the  odiervaiuUesXj.  A'jis 
a  ftuzy  set  which  can  be  obtained  fay  implying  f.  in  the  sense  of  fiizzy  set  and  possibility  theocy,  to  dK 
Aj's(j#i).i.c. 

'®'*i.  liA'i(xi)*o>in[liAi(xi).  «P  niin  liAj(*j)]  (3) 

When  dw  Aj's  are  fuzzy  intervals  and  f  is  monottHuc  with  re^tect  to  each  variable  and  can  be  ejqnessed 
in  terms  of  arithmetic  (qierations,  the  A'l's  are  fiizzy  intervals  which  can  be  easily  computed  using 
results  of  fiizzy  arithmetic  ;  see  Dubois  and  Prade  (1985,  1987).  This  extends  die  hex  that,  for 
instance,  in  the  above  exanqile  the  A'{'s  can  be  obtained  as  the  result  of  operations  <»  intervals  ; 
namely  A'j  ■  Aj  n(A3  e  A2).  A'2  ■  Aj  i^fAj  e  A^),  A'3  ■  A3  r»(Ai  •  A2),  where  the  circled 
symbols  are  used  for  denoting  the  extension  of  arithmetic  operations  to  intervals.  Indeed  fiizzy 
arithmetic  generalizes  interval  arithmetic.  Note  thu  the  refinement  is  obtained  in  (2)  in  one  step,  in  the 
sense  that  refined  A  j's  cannot  enable  us  to  obtain  a  more  restrictive  A'j.  This  can  be  easily  checked ; 
indeed,  taking  n  «  2  for  notational  convenience,  we  have 

““(UAtCxi).  “P nrin(UR(xi a2).  liA  2(x2))) 

*2 

■  sup  min(HAi(xi).  liR(xiJt2).  ^  niin(jiAi(xi).  liR(xiPt2)).  M2(='2)) 

■  4A'i(xi)  since  obviously  mm(HAi(xi).  MR(xiJf2))  ^  sup  nnnfjiAjCxi).  W(xi^2)) 

In  fact,  (2)  can  be  viewed  as  a  particular  case  of  the  general  qipniach  10  approximate  reasoning  initiaiwt 
in  Bellman  and  Zadeh  (1977)  and  developed  in  Zadeh  (1979),  namely,  all  the  pieces  of  information  are 
conjunctively  combined  and  then  the  result  is  projected  on  the  domain  of  the  variablefs)  in  which  we 
are  interested.  Indeed  (2)  can  be  equivalently  rewritten 

Vi.  Vxi.  PA'i(xi)  -  supxj  minOiR(xi.....Xn).PAi(xi) . HAjC*!) . 4An(*n))  (4) 

j-lm;  j^ 

In  case  of  several  relations  Ri^  the  cmnbinatioii^irojection  method  leads  to  the  foUowing 
updating  scheme  where  the  Rj^'s  ate  replaced  by  their  cylindrical  extensions  when  tiiey  do  not  involve 
allthevariatdes 

Vi,Vxi.  jiA'i(xi)-  supxj  mii'(mink.ijjiR^(xi.  ....xn),  minj.ij,  ltA;(xj))  (5) 

j-lm ;  jM. 

Sminic.ijr[min(HAi(xi).  suPxj  minOtR^(xi . Xn).  min  HAjCJ'j)))!  (6) 

j-l4i;jiri  j-l4i;j^ 

Tiie  inequality  (6)  expresses  that  if  we  take  into  account  each  R^  separately  in  the  refinement  {nocess, 
we  are  not  sure,  even  if  we  iterate  the  procedure  as  in  the  Waltz  algorithm,  of  obtaining  die  most 
accurate  refinement  for  each  variable  range.  Ifowever,  what  is  got  by  (6)  is  obviously  valid  and  mote 
easy  to  compute  in  genetaL 

Note  that  in  case  of  binary  relatioas,  the  Waltz  pracadure  (Le.  the  sepanie  preceninf  of 
the  Rjc's)  yields  the  most  accurate  result  given  by  (5).  provided  there  is  at  most  one  relation  R|( 
between  any  pair  of  variables  (xi,xj)  and  that  there  is  no  cycle  in  the  non^ented  graph  whose  nodes 
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conespond  to  the  variables  and  edges  u  die  binaiy  relations.  Indeed,  for  instance  with  n  «  3  and  two 
lelatkns,  we  have 

niin(HAj(xi).  supx2jc3  niin0iR(xi,x2).  HR'(X2A3),  HA2^*2).  UA3(*3)) 

» inin(HAi(xi)3upx2  inin0iR(xipt2).™o(KA2<*2)4«>Px3  ®fo(iiR'(x2.*3)*l^A3(»3))))) 


2^  •  Far-zv  •qmaliHts  and  ineaaalitita 


hi  this  subsection,  we  consider  pardcular  fozzy  relations  which  are  of  interest  in  practice 
for  qualitative  reasoning.  Approximate  equalities  or  strong  inequalities  (e.g.  'much  greater  than")  are 
examples  of  binaiy  fuzzy  relations  which  can  be  easily  handled  using  fuzzy  arithmetic  techniques. 
Indeed  an  approximate  equality  can  be  modelled  by  a  fuzzy  relation  E  of  the  form  |X£(x,y)s 
^L(Ix  -  y0>  for  instance 

5  +  e  -lx  -  yl 


Vx,  Vy,  PE(’‘<y)  =  max(0,  niin(l 


1  if  bt  -  yl  S  8 

-))-  (oifbc-yliS  +  e 

5  +  e-lx-yl 

- otherwise 


(8) 


where  8  and  £  are  respectively  positive  and  strictly  positive  parameters  which  modulate  the 
approximate  equality.  Then  the  approximate  equality  of  variables  X  and  Y  (in  the  sense  of  E)  will  be 
written  under  the  foim  of  the  equality 

X-Y-L  (9) 

with  the  following  intended  meaning ;  the  possible  values  of  the  difference  X  -  Y  are  restricted  by  the 
fuzzy  set  L.  Here  L  is  a  fuzzy  interval  centered  in  0,  i.e.  L  « -L  since  or  if  we  jxefer 

ltE(x.y)  »  iiE(y.x).  Similarly  a  strong  inequality  can  be  modelled  by  a  relation  I  of  the  form 

W(x.y)  *  Hk(*  *  y)>  for  instance  ,  ^  , 

x-v-X  r  iifx^y  +  ^  +  p 

Vx,  Vy,  Hj(x,y) »  max(0,  min(l, - ))  OifxSy+X  (10) 

P  x-y-X 

- otherwise 

P 

where  X  2  0  and  p  >  0.  The  constraint  X  is  much  greater  than  Y'  (in  the  sense  of  I)  can  then  be 
written 


X-Y-K  (11) 

where  K  is  a  fozzy  interval  such  that  K  «  [K,-««*)  (with  tt{K,-«-M)(0  *  PK(^))>  i-o-  K  identifies 

s5t 


itself  u  die  set  of  values  equal  cr  greater  than  a  value  restricted  by  K. 


If  we  know  for  instance  that  Xi  is  approximately  equal  to  X2'  (i.e.  X^  -  X2  ■  L)  and  that 
X2  is  tnuch  greater  than  X3’  (i.e.  X2  *  X3  ■  K),  we  can  deduce  that 

Xi-X3-L«K 

where  •denotes  the  addition  extended  to  fuzzy  intervals^  (see  Dubois  and  Prade  (1980, 1987)).  It  can 

l.LMOdHMiaiiieexiensioaafaaariiliiiiMicopmjan  *  wteayaaaofteiMlUiie.  ebdcCucdby 

Waridw  Wi[K)(t)  ■  lep  URW-WlNa  *isil»aiUitianandK«dLmtnpeaoidt 

M*l  »^t) 

lapnsMad  by  the  ibaciiaai  of  the  endpoints  of  their  pmlWddsB,  it  can  he  praved  that  (ki.k2.k3,k4)  •01,12.13, 
I4)  ■  (ki  ^  ll.  k2  *  b*  *^3 *7- ^ O4  nr  Ij  nay  be  eqoni  10 -OT  or  ««•). 
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be  proved  that  it  means  that  it  is  certain  that  Xi  2  X3  X  -(S  E)  and  that  the  value  of  the  difference 
X]  -  X3  belongs  to  L  e  K  at  the  degree  1  as  soon  as  X^  2  X3  -f  X  p  -  5.  See  Hgure  1.  Then 
depending  on  the  respective  values  of  the  parameters,  X^  is  still  greater  than  X3  (but  may  be  not  as 
muchasX2withrespecttoX3)(ifX>5-t-£),  orweareonly  surethatXi  is  not  much  smaller  than 
X3  (if  X  -)■  p  <  5).  Moreover,  if  we  know  that  X3  «  A3,  we  shall  get 

Xj  *  A'l »  A3  9  L  e  K 

This  is  a  particular  case  of  (7)  where  R  *  E,  R'  *  1,  A2  “  (-~,+»)  ■  Aj. 


2.3  •  Linear  constraints 

Another  worth-considering  particular  case  of  the  general  problem  presented  in  2.1  is  the 
one  of  linear  systems  of  constraints.  For  sake  of  simplicity,  we  only  briefly  discuss  linear  systems 
with  two  variables  and  two  constraints  of  the  form 

aiXi  +  biX2  =  A3 

a2Xi  +  b2X2  =  A4 

where  A3  and  A4  are  fuzzy  sets  of  teal  numbers,  and  the  other  coefBcients  are  real  numbers.  Note  that 
each  of  these  constraints  implicitly  defines  a  fuzzy  relation  which  restricts  the  possible  values  of  the 
pair  (Xi,X2).  Provided  that  aib2  -  a2bi  *  0,  we  can  deduce,  using  (3),  that 

b2A3  e  biA4  a2A3  ©  aiA4 

Xi»A'i« -  ;  X2  =  A2» -  (12) 

aib2  -  a2bi  a2bi  -  aib2 

with  Ai  »  A2  *  (-~,+«>) ;  see  the  footnote  1  for  the  de&iition  of  the  extended  difference  ©  and  of  the 
produa  of  a  fuzzy  quantity  by  a  scalar.  If  the  constraints  are  changed  into  ajXi  -f  biX2<*X3 
and  a2X2  b2X2  ••  X4,  with  X3  •>  A3  and  X4  <•  A4,  the  ranges  of  possible  values  of  X3  and  X4  are 
respectively  updated  into  A '3  =  A3  nfajA'i  ®  1*1^  2)  1**®  A'4  »  A4  n(a2A'i  ©  b2A'2). 

More  generally,  the  coefficients  in  linear  systems  may  be  ill-known.  Then  direct  extensions 
of  (12)  can  still  be  used  where  the  aj's  and  bj's  are  replaced  by  fuzzy  quantities  and  where  we  use  the 
product  and  the  quotient  defined  in  fuzzy  arithmetics.  However  in  that  case  we  get  ranges  which  are 
still  valid  but  may  be  larger  than  the  actual  ranges.  This  is  due  to  the  interactivity  constraint  which 
requires  that  the  values  of  a^  or  bj  should  be  the  same  at  the  numerators  and  the  denominators  in  (12), 
evoi  if  the  coefficients  are  ill-known,  and  which  is  forgonen  in  a  straightforwaitl  calculatioa.  This 
interactivity  constraint  should  be  taken  into  account  for  obtaining  the  actual  ranges.  See  Dubois  (1987) 
for  a  general  discussion  of  fuzzy  linear  programming. 


Standard  quaHtadve  reasomng  distinguisbes  between  values  which  are  strictly  negative  (-)> 
zero  (0)  or  strictly  positive  (+),  and  is  based  on  die  exploitation  of  the  following  tables  for  die  addidon 
and  the  product 


where  ?  denotes  the  completely  unknown  value  corresponding  to  the  range  (-«o,+oo).  However,  if  we 
know  for  instance  that  =  +  ;  X3  =  +  ;  Xj  +  X2  =  X3 

we  can  only  deduce  X2  =  ?  (while  if  Xi  =  0,  we  get  X2  *  +).  Another  simple  example  of  the 
undesirably  limited  representation  power  of  the  above  calculus  is  the  following 
if  Xi  »  +  and  X2  =  +  then  X3  =  Xj  +  X2  ••  + 

then  the  fact  that  X3  >  X^  and  X3  >  X2  is  forgotten.  These  kinds  of  ambiguities  could  be  removed,  if 
a  more  precise  knowledge  about  the  orders  of  magnitude,  which  is  often  available,  could  be  modelled. 
Indeed  we  have  in  the  general  case  for  the  first  above  exan^le 

Xi  »  Aj  :  X2  =  A2  :  X3  »  A3  ;  Xi  +  X2  »  X3 
from  which  we  deduce  X2  »  A'2  =  A2  n(A3  ©  Ai). 


This  kind  of  thing  still  can  be  done  in  an  approximate  way  when  the  Aj’s  are  required  to 
belong  to  a  prescribed  set  of  labels,  such  as,  for  instance  :  negative  large  (ML),  negative  medium 
(NM),  negative  small  (NS),  zero  (0),  positive  small  (PS),  positive  medium  (PM),  positive  large  (PL), 
unknown  (?).  These  labels  can  be  represented  by  fuzzy  intervals  such  as  the  ones  pictured  in  Figure  2. 

They  form  a  (fuzzy)  partition  of  the  real  line  in  some  sense. 

1 


0 

Rpire2 

The  condition  requested  to  build  a  meaningful  qualitative  calculus  are  twc^old : 

Cl .  The  advantage  of  qualitative  reasoning  is  linked  to  the  existence  of  symbolic  calculation  tables 
such  as  the  ones  above.  Such  tables  should  be  kept  when  absolute  orders  of  magnitude  are 
introduced. 

C2.  The  calculus,  even  qualitative,  should  remain  consistent  with  the  real  line  and  the  operations  of 
the  real  line  of  which  it  is  an  approximation. 

Standard  qualitative  reasoning  trivially  meeo  these  requirements.  However  going  beyond  the  four 
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symbols  0,  +,  ?  may  lode  challenging.  Indeed  the  closnre  property  of  the  taUe  seems  to  be 
incoaqratible  with  condition  C2.  For  instance  let  9  be  the  totally  ordered  set  of  symbds  (NL,  NM, 
NS,  0.  PS.  PM,  PL}  ;PS  9  PS  ■  PM  looks  reasonable  at  first  sight  But  PS  is  of  the  form  ]0ji]  and 
PS  9  PS  *  ]0,2a]  w  PM  >  [a,b].  Moreover  lim  nPS  «  ?.  Hence  results  obtained  ficom  die  addition 

n  >■»— 

taUe  built  from  ^  such  that  PS  9  PS  >  PM  is  inconsistent  with  die  addition  on  the  reals. 

It  does  not  mean  that  qualitative  reasoning  based  on  absolute  orders  of  magniowte  is  a 
utopia.  Interpreting  orders  of  magnitude  as  intervals  or  fuzzy  intervals  apparently  forbids  the  closure 
prcqierty  of  calculation  tables.  But  the  closure  property  can  be  preserved  on  subsets  of  $  containing 
adjacent  elements,  instead  of  §  itself,  provided  that  we  look  fer  the  best  approximatimi  On  the  sense  of 
inclusion)  of  Sj  9  ^  by  means  of  unions  of  adjacent  s^'s,  i.e.  Sj  9  »  ^  u  (s^).  Note  that  the 

keK 

introduction  of  the  symbol  ?  in  the  usual  qualitative  tables  meets  the  same  purpose,  that  is 
+9-  ^  (-,0,-t-}  =  ?.  What  is  proposed  is  just  a  generalization  of  the  way  the  symbol  ?  appears. 

The  example  of  Figure  2  leads  to  consider  the  following  term  set  ^  =  (NL,  NM,  NS,  0, 
PS.  PM.  PL.  [NLJ«d].  [NMJiS],  [NSJ»S]....  [NL4>M],  [NMJ?L],  ?}  where  [si.sj]  »  {skl 
Si£si(^Sj}forsiE  §  -{0),  Sj  e  S  -{0),  Si  <  sj.  Of  course  +  » [PSJPL]  and  -  *»  [NU^S].  Note  that 

if  S  has  n  elements  distinct  from  0  then  ■>  (n -Kn  •  1)  1)  -f  1  ■  1— -2? .  +  1  elements.  Here 

2 

1*^1  a  22,  for  instance.  This  size  is  not  so  large  for  contemporary  computers. 


s 

PM 

PL 

PM- 

PM+ 

+ 

5 

El 

■f 

•f 

PM 

mg 

El 

[S3 

[SB 

PL 

El 

la 

El 

El 

El 

Bl 

PM- 

* 

Gsa 
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[SB 

PM-" 

inn 

El 

^^3 

rya 

mo 

♦ 

* 

(23 

El 

ESS 

Table  2 :  PM"  -  (PS  J»M] ;  PM+  -  [PMJPL] 

In  Table  2  is  part  of  the  addition  uble  (for  strictly  positive  symbols),  without  any 
assumption  regarding  the  model  of  PS.  PM.  PL  (except  that  they  are  adjacent).  Note  that  this  Table 
corresponds  to  an  associative  operation,  when  restricted  to  positive  values.  However,  it  is  no  longer 
possible  to  preserve  associativity  on  the  whole  table.  This  is  due  to  the  approximation  procedure  since 
associative  operations  remain  associative  when  extended  to  intervals  or  fiizzy  intervals.  For  instance 
witfaNL-*Pl.(NL9PM)9PS  -  •9PS  >  [NL,PS].  while  NL  9(PM  9  PS)  «  NL  9  PM*^  -  ?. 
However  this  lack  of  usociativity  does  hot  prevent  to  use  this  approach,  since  the  ranges  which  are 
obtained  will  be  always  valid  even  if  they  may  be  too  large  with  reflect  to  the  availalrie  knowledge. 
Moreover,  we  may  try  to  perform  operations  in  a  way  where  no  information  is  lost 
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The  Addition  law  can  be  improved  (with  regard  to  the  precision  of  its  tefiilts  by  snbeetpient 
requirements  for  instance  K  «  PS  ■>  1^*,  which  forces  PS  >  K)4t].  PM  « [a,b]  with  2a  ^  b.  Note  diiat 
it  is  not  necessary  to  use  fhi^  intervals.  Adjacent  intervals  can  do  the  job.  However  there  will  be 
discontinuity  problems  when  the  (real)  values  of  variables  cross  the  boundaries  of  the  intervals 
modeling  the  symbol  Only  fiizzymtervals  can  cope  widi  these  problems. 


4  .  Fuzzy  rcifltiang  and  orders  of  maynitude 

Orders  of  magnitude  can  be  expressed  in  an  absolute  way  in  terms  of  labels  such  as 
"small”,  "medium"  or  "large"  which  can  be  represented  by  fuzzy  intervals,  as  said  in  section  3.  They 
can  also  be  handled  in  a  relative  way  by  means  of  relations.  This  is  the  topic  of  the  present  section. 
Raiman  (1983,  1986)  has  proposed  a  fonnal  system  for  order  of  magnitude  reasoning  with  three 
binary  operators  :  Ne  (for  'negligible  in  relation  to'),  Vo  (fen  'close  to'),  and  Co  (for  'comparable  to'). 
Inference  rales,  which  can  be  justified  from  a  Non-Standard  Analysis  point  of  view,  describe  how 
these  operators  woik  together.  See  Bourgine  and  Raiman(1986)  for  an  application  in  macroeconomics. 
In  the  following,  we  discuss  the  modeling  of  these  operators  in  terms  of  fuzzy  relations. 


The  idea  of  closeness  seems  to  be  naturally  captured  by  an  approxinuue  equality  relation. 
Raiman  (1986)  relates  the  ideas  of  closeness  and  of  negligibility  in  the  following  way :  'x  is  close  to  y' 
is  equivalent  to  '(x  -  y)  is  negligible  in  relation  to  y'.  In  other  words,  'x  is  negligible  in  relation  to  y'  if 
and  only  if  'x  +  y  is  close  to  y'.  If  we  use  an  approximate  equality  of  the  form  liE(x,y) «  jil(Ix  -  yl)  (as 
in  2.2)  for  modelling  'close  to',  the  above  equivalence  would  lead  to  a  definition  of  'negligible'  which 
would  not  be  relative  (since  l(x  +  y)  -  yl  =  Ixl  does  not  depend  on  y),  but  absolute.  It  can  be  avoided  by 
defining  the  fiizzy  relation  'Vo'  in  terms  of  a  quotient,  i.e. 

liVoC’s-y)  *  )  (13) 

y 


where  the  characteristic  function  nj^  is  such  that  pm(1)  “  1  “d  1^M(*)  =  1^M(— )•  ’Th**® 

t 

liVo(*-y)  ■  l^VoCy-^)  “dM  is  a  fuzzy  interval  which  restricts  values  which  are  around  1  and  which  is 

equal  to  its  "inverse",  i.e.  M  »  —  (however  we  have  not  »  1 !).  Then  it  leads  to  define  the  extent  to 

M 


which  X  is  negligible  in  relation  to  y,  by 

liNe(*.y)  ■  (W) 

y 


The  combination/^jcction  method,  used  in  2.1,  enables  us  to  petficitm  the  compoeition  of 
Vo  or  of  Ne  with  itself,  or  of  Vo  with  Ne.  The  following  results  are  easy  to  establish  2 


2.  Warning ;  in  interval  arithmetic  and  more  genanily  in  fuzzy  arldnnaiic,  the  pndna  MM  ia  equal  to  M^  if  and  only  if 
M  is  eidier  positive  (Le.  |iM(s)  >  0  «  x  2  0)  or  negative  O^e.  iqt((x)  >0^xS0).  Herein  pneiice  M  is  pneiiive. 
batnoi(M-l). 
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snp  it]inOiVo(^y)>  (iVo(y«*))  ■  )  *  >^Vo(w)  (15) 

y  z 

sup  iBin(MNe(**y)*  t^NeCy.*))  •  S  ^NeCw)  (W 

y  z 

X*^Z 

(up  Biin(nvo(*.y).  4N«(y.*»  ■  IHMCM-l)  •  1]( - )  ^  MNe(w)  d^) 

y  * 

X 

sup  min()ivo(x  +  y,  z),  )iNe(y>x))  ■  ^  MVo(***)  (18) 

y  * 

They  conespond  to  the  ((dlowing  inference  rules  proposed  by  Raiman  (198^  (for  sake  of  brevity,  here 
we  only  discuss  a  part  of  the  30  rules  used  in  the  formal  system) 

6)  (x  Vo  y)  A  (y  Vo  z)  -*  (xVoz)  ;  (n)  (xNcy)A<yNez)  -»  (xNez) 

6ii)  (x  Vo  y)  A  ^  Ne  z)  -*  (x  Ne  z)  ;  (hr)  ((x  +  y)  Vo  z)  a  (y  Ne  x)  -»  (x  Vo  z) 

The  fuzzy  relation  approach  shows  that  several  of  these  rules  are  only  "qualitatively  valid".  Indeed  in 
(15),  the  fact  that  MM  is  a  fuzzy  set  which  contains  M  miiTocs  the  intuitively  satisfying  lack  of 
transitivity  of  the  fuzzy  relation  Vo,  strictly  ^tealdng.  By  contrast,  as  shown  by  (15),  the  relation  Ne  is 
transitive.  The  repeated  use  of  the  formal  rules  (i),  (iii)  or  (iv)  without  contnd  can  lead  to  dubious 
conclusions  in  a  way  similar  to  sorites  such  as  die  bald  man  paradox  (Le.,  adding  an  hair  to  a  bald  man 
leaves  him  bald,  but  if  we  repeat  the  addition...).  The  tesnla  of  the  conqiosition  of  fuzzy  relations, 
such  as  (15)-(18),  are  easy  to  compute  in  terms  of  sinqile  fuzzy  arithmetic  operations  on  M.  The 
fuzzy  relation  calculus  enaUes  us  to  reason  about  closeness  and  negligibility  in  a  rigorous  way  without 
limitations  on  the  chaining  means  of  conmd  techniques. 

N.B.  1  Inference  rules  expressing  the  cmnpatibility  of  the  relations  with  respect  to  arithmetic 
operations,  such  as  (x  Vo  y)  a  (z  Ne  t)  -»  xz  Ne  yt  can  be  also  discussed  in  our  framework.  Indeed  it 
can  be  proved  that 

supx,y,z,t  niin(itvo(*.y).liNe(^*))-li[M(M-l)*l](-^^)251iNe(«»v)  (19) 

u»xz;v^  V 

Again  we  see  that  the  rule  is  only  "qualitatively  valid”,  Le.  xz  may  be  slightly  less  negligible  with 
respect  to  yt  than  z  in  relation  to  t.  Alternative)  y,  we  could  ooiqpote  what  is  the  possibility  diat  u  is  not 
negligible  (in  the  sense  of  Ne)  with  respect  to  v,  from  (19). 

NJ.2  Note  that  we  have  only  an  approximate  equality  between  ltiqe(’^y)  and  lti4e(*^y)  (1^)  • 
a  perfect  equality  could  be  recovered  by  motfifying  (14)  faiio  |iii^x,y)  ■  *■)• 

y-» 

NJ.  3  Raiman  (1985)  makes  use  of  a  third  relation  Co  wUdi  is  sach  that  if  x  Vo  y,  then  x  Co  y 
and  expresses  that  two  values  have  the  tame  sign  and  the  same  order  of  mngnitode.  We  may  nanfine  to 
define  Co  initiation  to  Vo  and  Ne  in  dUferant  ways,  frir  inatance  by  caressing  that  xCoy  iff 
Vz,  X  Ne  z  «•  y  Ne  z,  following  Raiiaan  (1985).  Another  wKy  would  be  to  state  that  x  Co  y  iff 
not((x  Ne  y)  A  (y  Ne  x)]  in  the  sente  of  some  ftizzynegatioo  ■  lobechoaeainrdationwidi  (sm  ^ 

order  to  have  inax(a()iM(l  u)],  nbiM(l  ■<>  i-.)l)  2  |tM(**)>  Vn  On  order  »  gnaramee  mco  ^  HVo)- 

u  .  . 
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Odier  tools,  not  presented  here,  which  have  been  also  developed  in  fiiz^  setor  ponihOity 
theory,  may  tun  to  be  useful  in  qualitative  reasoning.  Qualitative  descriptions  of  the  dependency 
between  variables  of  the  foim  "the  more  (or  die  less)  Xi  is  and. . .  and  is  Aq,  die  ihoie  (or  the 
less)  Y  is  B",  where  A|, ...,  Ag  and  B  are  gradual  properties,  can  be  conveniently  represented  (by 
means  of  a  special  kind  of  fuzzy  relation)  and  dealt  with  in  the  framework  of  fuzzy  lo^  as  recently 
shown  in  Dubois  and  Prade  (1988):  Such  gradual  rules  naturally  provide  a  qualitative  description  of 
the  behavior  of  systems.  For  instance,  with  n  «  2,  Aj  «  large',  A2  ■  'small',  B  ■  large'  and  the 
hedges  "the  more...  the  more",  we  express  that  "if  Xj  increases  and  X2  decreases  then  Y  increases" 

(the  nature  of  the  increasingness  or  of  the  dccreasingness  can  be  modulated  throng  a  proper  choce  of 

tiAi.iiA2“dPB)- 

Besides,  a  methodology  for  the  control  of  complex  dynamical  systems  by  means  of  fuzzy 
expert  rules  which  provide  a  qualitative  description  in  terms  of  fuzzy  sets  of  the  relation  between  action 
variables  and  observable  state  variables,  was  settled  more  than  ten  years  ago  (Mamdani  and  Assilian, 
197S) ;  see  Sugeno  (198S)  for  an  overview  of  existing  applications.  People  in  Artificial  Intelligence 
have  also  considered  the  problem  of  qualitative  control  recently  (e.g.  docksin  et  Morgan,  1986). 

The  intended  purpose  of  this  short  communication  is  to  point  out  that  fuzzy  set  and 
possibility  theory  can  offer  valuable  tools  for  qualitative  reasoning  problems.  In  particular 
"commonsense"  arithmetic  reasoning  (e.g.  Simmons,  1986)  can  be  easily  bandied  using  fuzzy 
intervals  and  fuzzy  comparison  relations.  This  framework  is  especially  useful  for  interftcing  numerical 
data  and  symbolic  informatioiL 
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